pdftochat

PDFToChat

Chat with your PDFs in seconds. Powered by Together AI and Chroma.

Tech Stack · Deploy Your Own · Common Errors · Credits · Future Tasks

## Tech Stack - Next.js [App Router](https://nextjs.org/docs/app) for the framework - Mixtral through [Together AI](https://togetherai.link/?utm_source=pdftochat&utm_medium=referral&utm_campaign=example-app) inference for the LLM - [Chroma Cloud](https://www.trychroma.com/) for vector search (hybrid dense + sparse via Qwen & SPLADE) - [LangChain.js](https://js.langchain.com/docs/get_started/introduction/) for the RAG code - [Bytescale](https://www.bytescale.com/) for the PDF storage - [Vercel](https://vercel.com/) for hosting and for the postgres DB - [Clerk](https://clerk.dev/) for user authentication - [Tailwind CSS](https://tailwindcss.com/) for styling ## Deploy Your Own You can deploy this template to Vercel or any other host. Note that you'll need to: - Set up [Together.ai](https://togetherai.link/?utm_source=pdftochat&utm_medium=referral&utm_campaign=example-app) for the LLM - Set up [Chroma Cloud](https://www.trychroma.com/) for vector search - Set up [Bytescale](https://www.bytescale.com/) for PDF storage - Set up [Clerk](https://clerk.dev/) for auth - Set up [Vercel](https://vercel.com/) for hosting + Postgres (or use [Neon](https://neon.tech/)) - (Optional) Set up [LangSmith](https://smith.langchain.com/) for tracing See `.env.example` for a list of all the required environment variables. You will also need to prepare your database schema by running `npx prisma db push`. ### Chroma Cloud This project uses [Chroma Cloud](https://www.trychroma.com/) for hybrid vector search. Embeddings are generated automatically by Chroma Cloud using **Qwen** (dense) and **SPLADE** (sparse), combined via Reciprocal Rank Fusion (RRF) at query time. No local embedding model is needed. 1. Sign up at [trychroma.com](https://www.trychroma.com/) and create a database. 2. Copy your credentials and set the following environment variables: ```ini NEXT_PUBLIC_VECTORSTORE=chroma CHROMA_API_KEY= # Your Chroma Cloud API key CHROMA_TENANT= # Your tenant ID CHROMA_DATABASE= # Your database name ``` Collections are created automatically per document — no manual index setup is required. ## Common errors - Check that you've created an `.env` file that contains your valid (and working) API keys. - Check that `NEXT_PUBLIC_VECTORSTORE` is set to `chroma` and your Chroma credentials are correct. - Check that you've run `npx prisma db push` to create the `Document` table in Postgres. - Check that you've added a credit card on Together AI if you're hitting rate limiting issues due to the free tier. ## Credits - [Youssef](https://twitter.com/YoussefUiUx) for the design of the app - [Mayo](https://twitter.com/mayowaoshin) for the original RAG repo and inspiration - [Jacob](https://twitter.com/Hacubu) for the LangChain help - Together AI, Bytescale, Chroma, and Clerk for sponsoring ## Future tasks These are some future tasks that I have planned. Contributions are welcome! - [ ] Add a trash icon for folks to delete PDFs from the dashboard and implement delete functionality - [ ] Try different embedding models like UAE-large-v1 to see if it improves accuracy - [ ] Explore best practices for auto scrolling based on other chat apps like chatGPT - [ ] Do some prompt engineering for Mixtral to make replies as good as possible - [ ] Protect API routes by making sure users are signed in before executing chats - [ ] Run an initial benchmark on how accurate chunking / retrieval are - [ ] Research best practices for chunking and retrieval and play around with them – ideally run benchmarks - [ ] Try out Langsmith for more observability into how the RAG app runs - [ ] Add demo video to the homepage to demonstrate functionality more easily - [ ] Upgrade to Next.js 14 and fix any issues with that - [ ] Implement sources like perplexity to be clickable with more info - [ ] Add analytics to track the number of chats & errors - [ ] Make some changes to the default tailwind `prose` to decrease padding - [ ] Add an initial message with sample questions or just add them as bubbles on the page - [ ] Add an option to get answers as markdown or in regular paragraphs - [ ] Implement something like SWR to automatically revalidate data - [ ] Save chats for each user to get back to later in the postgres DB - [ ] Bring up a message to direct folks to compress PDFs if they're beyond 10MB - [ ] Use a self-designed custom uploader - [ ] Use a session tracking tool to better understand how folks are using the site - [ ] Add better error handling overall with appropriate toasts when actions fail - [ ] Add support for images in PDFs with something like [Nougat](https://replicate.com/meta/nougat)