Quarkus and LangChain4J - A Match Made in Heaven

10/20/24 • 62 min

An airhacks.fm conversation with Georgios Andrianakis (@geoand86) about: discussion on integrating langchain4j with quarkus for enterprise AI applications, similarities between LLM integration and microservice architecture, benefits of using Java and MicroProfile for AI development, explanation of AI services, chat memory, and tools in LangChain4J, importance of session management and fault tolerance in LLM applications, vector databases and embeddings for efficient information retrieval, RAG (Retrieve Augmented Generation) implementation in enterprise settings, Quarkus dev mode features for LLM experimentation, native image support with GraalVM, local inference possibilities with Java 21's Vector API and quantized models, challenges in prompt engineering and model selection, upcoming features in LangChain4J including Ollama tool support and improved result streaming, future developments in Java for AI and GPU support with Project Babylon, importance of enterprise-grade features like CI/CD, testing, and cloud deployment for LLM applications

Georgios Andrianakis on twitter: @geoand86

Georgios Andrianakis on twitter: @geoand86

Previous Episode

Why JVector 3 Is The Most Advanced Embedded Vector Search Engine

An airhacks.fm conversation with Jonathan Ellis (@spyced) about: discussion of JVector 3 features and improvements, compression techniques for vector indexes, binary quantization vs product quantization, anisotropic product quantization for improved accuracy, indexing Wikipedia example, Cassandra integration, SIMD acceleration with Fused ADC, optimization with Chronicle Map, E5 embedding models, comparison of CPU vs GPU for vector search, implementation details and low-level optimizations in Java, use of Java Panama API and foreign function interface, JVector's performance advantages, memory footprint reduction, integration with Cassandra and Astra DB, challenges of vector search at scale, trade-offs between RAM usage and CPU performance, Eventual Consistency in distributed vector search, comparison of different embedding models and their accuracy, importance of re-ranking in vector search, advantages of JVector over other vector search implementations

Jonathan Ellis on twitter: @spyced

Next Episode

Java, LLMs, and Seamless AI Integration with langchain4j, Quarkus and MicroProfile

An airhacks.fm conversation with Dmytro Liubarsky (@langchain4j) about: discussion on recent developments in Java and LLM integration, new features in langchain4j including Easy RAG for simplified setup, SQL database retrieval with LLM-generated queries, integration with graph databases like Neo4j, Neo4j and graphrag, metadata filtering for improved search capabilities, observability improvements with listeners and potential integration with opentelemetry, increased configurability for AI services enabling state machine-like behavior, the trend towards CPU inference and smaller, more focused models, langchain4j integration with quarkus and MicroProfile, parallels between AI integration and microservices architecture, the importance of decomposing complex AI tasks into smaller, more manageable pieces, potential cost optimization strategies for AI applications, the excitement around creating smooth APIs that integrate well with the Java ecosystem, the potential future of CPU inference and its parallels with the evolution of server infrastructure, the upcoming Devoxx conference,

Dmytro Liubarsky on twitter: @langchain4j