For the past few months, I've been building OxyJen, an open-source framework for building reliable, multi-step AI pipelines in Java. In most Java LLM projects, everything is still just strings. You build a prompt, make a call, and then parse your regex and wait if it works on the "almost-JSON" that comes back. It's brittle, untestable, and feels like the opposite of Java's "contract-first" philosophy.

OxyJen's approach is different. It's a graph-based orchestration framework(currently sequential) where LLMs are treated as native, reliable nodes in a pipeline, not as magical helper utilities. The core idea is to bring deterministic reliability to probabilistic AI calls. Everything is a node in a graph based system, LLMNode, LLMChain, LLM api is used to build a simple LLM node for the graph with retry/fallback, jitter/backoff, timeout enforcements(currently supports OpenAI).

PromptTemplate, Variable(required/optional) and PromptRegistry is used to build and store reusable prompts which saves you from re writing prompts.

JSONSchema and SchemaGenerator is used to build schema from POJOs/Records which will provide Json enforcements on outputs of these LLMs. SchemaNode<T> wraps SchemaEnforcer and validator to map LLM output directly to the classes. Enforcer also makes sure your output is correct and takes maximum retries.

Currently working on the Tool API to help users build their custom tools in Oxyjen. I'm a solo builder right now and the project is in its very early stage so I would really appreciate any feedback and contributions(even a small improvement in docs would be valuable).

OxyJen: https://github.com/11divyansh/OxyJen