# Supercharging LLM application development with LLM-Kit

[Grab](https://yomu.fyi/company/grab) · Boon Zhan Chew · Nov 29, 2024

## Summary

Grab developed LLM-Kit to resolve standardization, security, observability, and infrastructure provisioning bottlenecks across its generative AI development efforts. Submitting an initial application request automatically triggers project generation in GitLab, outputting a modular codebase along with Terraform definitions for Amazon EKS and ECR. The scaffolded architecture incorporates FastAPI, LangChain, OpenID Connect authentication helpers, and PGVector alongside HashiCorp Vault for credential management. Observability and quality tracking are built in through direct integrations with Datadog and LangSmith evaluations. By providing these standardized paved-road templates, the framework has onboarded hundreds of generative AI applications and saved teams an estimated 1.5 weeks of initial setup time.

## Takeaways

- LLM-Kit automates application bootstrapping by generating a repository scaffold, GitLab CI pipelines, and Terraform configurations for AWS infrastructure upon form submission.
- The default application structure standardizes dependencies using FastAPI, LangChain, Gunicorn, PGVector for vector storage, and HashiCorp Vault for secure token retrieval.
- Integrated tracing and evaluations via LangSmith and Datadog allow teams to monitor production LLM behavior and detect issues in real time.

**Tags:** [CI/CD](https://yomu.fyi/topic/ci-cd), [Developer Experience](https://yomu.fyi/topic/developer-experience), [Kubernetes](https://yomu.fyi/topic/kubernetes), [LLMs](https://yomu.fyi/topic/llm), [Observability](https://yomu.fyi/topic/observability)

[Read original post](https://engineering.grab.com/supercharging-llm-application-development-with-llm-kit)