OpenLIT 2.0

OpenLIT 2.0 is a handy, open-source tool you can host yourself. It makes creating AI applications much easier, especially when working with Generative AI and Large Language Models, or LLMs.

Key Features

Testing LLMs and Managing Prompts

OpenLIT lets developers try out different LLMs, organize and update prompts, and keep API keys safe. This makes creating AI applications efficient and secure.

Easy Monitoring

With just a simple line of code, OpenLIT watches over LLMs, vector databases, and GPUs from start to finish. This helps developers move AI features from testing to real use easily.

Analytics Dashboard

The platform has an analytics dashboard that keeps an eye on the health and performance of AI applications. It tracks important numbers, costs, and how users interact, giving a clear view of how well everything is working.

Tracking Costs

OpenLIT helps track costs for special and fine-tuned models using custom pricing files. This helps with budgeting and aligning with project needs.

Watching for Issues

A special monitoring dashboard helps spot and fix common problems quickly, keeping AI applications running well.

Managing Prompts

Prompt Hub lets you manage and update prompts, ensuring they are consistent and easy to access across applications.

Keeping API Keys Safe

API keys and secrets are kept safe and central, making AI applications more secure.

Trying Out LLMs

OpenGround lets developers test and compare different LLMs side by side, helping them make good choices.

Supported GenAI and LLM Libraries

OpenLIT works with many LLMs and vector databases, like:

• LLMs: OpenAI, Ollama, Cohere, Anthropic, GPT4All, Azure OpenAI, Mistral, HuggingFace, Amazon Bedrock, Vertex AI, Groq
• Vector DBs: ChromaDB, Pinecone, Qdrant, Milvus
• Frameworks: LangChain, LiteLLM, LlamaIndex, Haystack, EmbedChain

Connecting to Observability Platforms

OpenLIT can connect to different observability platforms, such as:

• Prometheus
• Otel Collector
• Grafana
• Jaeger
• New Relic
• Datadog
• Signoz
• Dynatrace
• OpenObserve
• Highlight.io

User Interface

The OpenLIT UI is clean and quick, built with Next.js and Clickhouse. It is easy to navigate and find information, thanks to feedback from AI developers.

Overall Usage

The platform shows real-time and past data on total requests, request duration, token usage, cost analysis, and more. It also has filters to sort results by costs, tokens, and other factors.

Database Setup

OpenLIT lets users add and switch between multiple ClickHouse databases, making it easy to manage different tasks with the same interface.

Privacy and Self-Hosting

OpenLIT values privacy, letting users see what the code does or host it themselves. This ensures transparency and control over the AI development process.

Community and Contributions

OpenLIT welcomes users to join and help shape the future of AI observability. By sharing ideas, suggesting features, and contributing to the platform, users can help make AI observability better for everyone.