Skip to content

Chat with your data - AI data analysis and visualization on CSV, Postgres, MySQL, Snowflake, SQLite...

License

Notifications You must be signed in to change notification settings

RamiAwar/dataline

Repository files navigation

πŸ’¬ Chat with your data using natural language πŸ“Š

Introducing DataLine, the simplest and fastest way⚑️ to analyze and visualize your data!
Generate and export charts, tables, reports in seconds with DataLine - Your AI-driven data analysis and visualization tool πŸ€“

🍿 Watch a quick demo

DataLine logo


Index

Who is this for?

Technical or non-technical people who want to explore data, fast. ⚑️⚑️

It also works for backend developers to speed up drafting queries and explore new DBs with ease. 😎

It's especially well-suited for businesses given its security-first πŸ”’ and open-source πŸ“– nature.

What is it?

DataLine is an AI-driven data analysis and visualization tool.

It's privacy-focused, storing everything on your device. No ☁️, only β˜€οΈ!

It hides your data from the LLMs used by default, but this can be disabled if the data is not deemed sensitive.

It can connect to a variety of data sources (Postgres, Snowflake, MySQL, Azure SQL Server, Microsoft SQL Server, Excel, SQLite, CSV, sas7bdat, and more), execute queries, generate charts, and allow for copying the results to build reports quickly.

Where is it going?

For now, we're trying to help people get insights out of their data, fast.

This is meant to enable non-technical folks to query data and aid data analysts in getting their jobs done 10x as fast.

But you can still influence the direction we go in. We're building this for you, so you have the biggest say.

Feature Support

  • Broad DB support: Postgres, MySQL, Snowflake, Excel, CSV, SQLite, and more
  • Generating and executing SQL from natural language
  • Ability to modify SQL results, save them, and re-run
  • Better support for explorative questions
  • Querying data files like CSV, Excel, SQLite, sas7bdat (more connection types)
  • Charting via natural language
  • Modifying chart queries and re-rendering/refreshing charts
  • Dashboards and triggers
  • Knowledge base and 'trainable' examples (flavor of RAG)
  • More advanced charting options (bubble, stacks, etc.)

With a lot more coming soon. You can still influence what we build, so if you're a user and you're down for it, we'd love to interview you! Book some time with one of us here:

Getting started

There are multiple ways of setting up DataLine, simplest being using a binary executable. This allows you to download a file and run it to get started.

A more flexible option is using our hosted Docker image. This allows you to setup authentication and other features if you need them.

Windows

Head over to our releases page, and open the most recent one. There you should find a windows-exe.zip file. Download it, unzip it, and run the DataLine.exe file.

You might get a "Windows protected your PC" message, which is normal (more info -> run anyway). Finally, open http://localhost:7377/ in your browser.

Mac

Homebrew

# install dataline
brew tap ramiawar/dataline
brew install dataline

# run dataline
dataline

If you don't like Homebrew, a binary can be found in the latest release!

DataLine should then be running on port 7377 accessible from your browser: http://localhost:7377

Linux

You can use Homebrew, see the Mac section.

You may also wish to use the binary instead, to do so, follow the instructions in the Windows section, and use the dataline-linux.tar.zip file instead.

Docker

You can also use our official docker image and get started in one command. This is more suitable for business use:

docker run -p 7377:7377 -v dataline:/home/.dataline --name dataline ramiawar/dataline:latest

You can manage this as you would any other container. docker start dataline, docker stop dataline

For updating to a new version, just remove the container and rerun the command. This way the volume is persisted across updates.

docker rm dataline
docker run -p 7377:7377 -v dataline:/home/.dataline --name dataline ramiawar/dataline:latest

To connect to the frontend, you can then visit: http://localhost:7377

Upgrading

Upgrade Windows

Same as installation, just replace old exe with new exe! Your data will still be there across versions.

Upgrade Mac

Homebrew (retains your data, don't worry about that!)

brew update && brew upgrade dataline

If you don't like Homebrew, a binary can be found in the latest release! Data will still be retained there as well.

Upgrade Linux

If using Homebrew, same as above. Otherwise simply replace the old binary with the new one!

Upgrade Docker

For updating to a new version, just remove the container and rerun the command. This way the volume is persisted across updates.

docker rm dataline
docker run -p 7377:7377 -v dataline:/home/.dataline --name dataline ramiawar/dataline:latest

Running manually

Feeling spicy are we? 🌢️ There are a few things you should know. DataLine is split into two parts: the backend and the frontend.

The backend is a Python FastAPI server, and the frontend is a React app. The frontend also includes our landing page, so you need to set up an env var first!

Check the backend and frontend readmes.

Authentication

DataLine also supports basic auth πŸ”’ in self-hosted mode πŸ₯³ in case you're hosting it and would like to secure it with a username/password.

Auth is NOT supported ❌ when running the DataLine executable.

To enable authentication on the self-hosted version, add the environment variables AUTH_USERNAME and AUTH_PASSWORD while launching the service. βœ…

With Docker

Inject the env vars with the docker run command as follows: docker run -p 7377:7377 -v dataline:/home/.dataline --name dataline -e AUTH_USERNAME=admin -e AUTH_PASSWORD=admin ramiawar/dataline:latest

We plan on supporting multiple user auth in the future, but for now it supports a single user by default.

Startup Quest

Go through the following checklist to explore DataLine's features!

  • Create a sample database connection
  • Create a new chat and rename it
  • Start asking questions about your data and getting answers
  • Refresh the page and re-run some SQL queries
  • Click inside an SQL query, modify it, and save your modification for later!
  • Try to modify your sample DB connection and explore the connection editor page
  • Try asking for a chart!
  • To really challenge it, ask a question that is answered with multiple results (charts, tables, etc.) - example
  • Add a profile picture

Supported databases

See instructions file for more details.

Deployment

The one thing you must configure when deploying DataLine to a custom domain is CORS allowed origins. To do this, add the environment variable ALLOWED_ORIGINS (comma separated list of origins) to your domain name(s).

By default, it is set to http://localhost:7377,http://0.0.0.0:7377 to make it work with local Docker and local binaries.

For example, running the docker image on a remote server with IP 123.123.12.34:

docker run -p 7377:7377 -v dataline:/home/.dataline --name dataline -e ALLOWED_ORIGINS="http://123.123.12.34:7377,https://123.123.12.34:7377" ramiawar/dataline:latest

Excel Support

We support excel files, but they will have to conform to some structure for the time being. We also support multiple sheets - each sheet will be ingested as a separate table.

Right now, we will try to automatically detect the 'header row' and the first column based on some manual data processing (so as to keep things secure). This means that we might detect the wrong things if you have extra rows on top / logos / branding elements.

To ensure the best quality, make sure your first row is the column names, and remove any padding rows/columns from all the sheets. If any sheet fails, the import will fail.

Future improvements to this include optionally allowing LLMs to figure out what the header row is to reduce user effort.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Rami Awar
Rami Awar

πŸ’» 🎨 πŸ“– πŸš‡ πŸ“¦ πŸ“ πŸ›
anthony2261
anthony2261

πŸ’» πŸ€” πŸš‡ πŸ“¦ πŸ§‘β€πŸ« 🚧 πŸ›
Walusimbi Mahad
Walusimbi Mahad

πŸ’» πŸ€”
PΓ©ter Gyarmati
PΓ©ter Gyarmati

πŸ’» πŸ“–
Joe Haddad
Joe Haddad

πŸ’» πŸ“–
maryamalki
maryamalki

πŸ’» πŸ“–
Filip KatuΕ‘in
Filip KatuΕ‘in

πŸ’» πŸ›

This project follows the all-contributors specification. Contributions of any kind welcome!