Back to blog
Articles
Articles
August 3, 2023
·
5 min read

These Are The Updates To ChainForge

August 3, 2023
|
5 min read

Latest content

Customer Stories
4 min read

How Infobip Generated 220+ Knowledge Articles with Gen AI For Smarter Self-Service and Better NPS

Partnering with HumanFirst, Infobip generated over 220 knowledge articles, unlocked 30% of their agents' time, and improved containment by a projected 15%.
September 16, 2024
Articles
7 min read

Non-Technical AI Adoption: The Value of & Path Towards Workforce-Wide AI

Reviewing the state of employee experimentation and organizational adoption, and exploring the shifts in thinking, tooling, and training required for workforce-wide AI.
September 12, 2024
Articles
6 min read

AI for CIOs: From One-Off Use to Company-Wide Value

A maturity model for three stages of AI adoption, including strategies for company leaders to progress to the next stage.
September 12, 2024
Tutorials
4 min read

Building Prompts for Generators in Dialogflow CX

How to get started with generative features.
August 15, 2024
Announcements
3 min read

HumanFirst and Infobip Announce a Partnership to Equip Enterprise Teams with Data + Generative AI

With a one-click integration to Conversations, Infobip’s contact center solution, HumanFirst helps enterprise teams leverage LLMs to analyze 100% of their customer data.
August 8, 2024
Tutorials
4 min read

Two Field-Tested Prompts for CX Teams

Get deeper insights from unstructured customer data with generative AI.
August 7, 2024
Tutorials
5 min read

Optimizing RAG with Knowledge Base Maintenance

How to find gaps between knowledge base content and real user questions.
April 23, 2024
Tutorials
4 min read

Scaling Quality Assurance with HumanFirst and Google Cloud

How to use HumanFirst with Vertex AI to test, improve, and trust agent performance.
March 14, 2024
Customer Stories
4 min read

How Infobip Generated 220+ Knowledge Articles with Gen AI For Smarter Self-Service and Better NPS

Partnering with HumanFirst, Infobip generated over 220 knowledge articles, unlocked 30% of their agents' time, and improved containment by a projected 15%.
September 16, 2024
Articles
7 min read

Non-Technical AI Adoption: The Value of & Path Towards Workforce-Wide AI

Reviewing the state of employee experimentation and organizational adoption, and exploring the shifts in thinking, tooling, and training required for workforce-wide AI.
September 12, 2024
Articles
6 min read

AI for CIOs: From One-Off Use to Company-Wide Value

A maturity model for three stages of AI adoption, including strategies for company leaders to progress to the next stage.
September 12, 2024

Let your data drive.

Articles

These Are The Updates To ChainForge

COBUS GREYLING
August 3, 2023
.
5 min read

ChainForge is an IDE for prompt engineering and a number of important improvements were made to the tool.

Recently I created a diagram to map out the LLM application tooling landscape.

The tools listed employ the principle of prompt engineering in various ways ranging from a no-code to a pro-code approach. Virtually all the tools listed are focused on improving the end-user experience or prompt management and augmentation.

ChainForge has a different focus, in that ChainForge can be used to test how robust prompts are by evaluating the response from different models.

This is a practical approach of prompting multiple LLMs and comparing the different responses in a practical and hands-on way.

ChainForge is described as a visual programming environment for prompt engineering.

ChainForge can now be run from the public available URL, with no user account or installation required. Alternatively ChainForge can be installed locally with:

Considering the example below, which you can access by simply clicking here…the four LLMs (PaLM2, Claude, GPT4 & GPT3.5) are each given secret codes and is instructed not to divulge the secret codes.

A set of ambiguous prompts are then injected in order to trick the LLMs to include the codes in their response.

ChainForge has eight flow nodes to make use of, with a handy import and export feature. Export/import is convenient for sharing and storing flows and a flow can be instantly shared via the Share button.

The New Flow button initiates a new flow by discarding the existing flow. It would work much better if the New Flow button created a tab within the IDE. Where multiple flows can be open simultaneously and managed via tabs within ChainForge. A dark mode option will also be handy.

The LLMs available via ChainForge are OpenAI, HuggingFace, Google PaLM and OpenAI via Azure. You will need to have access to your own API keys for each LLM.

Something that makes the Vercel playground so attractive, is that you get to interact with LLMs without having your own log in credentials and access. As some of the LLMs still have very restricted access.

Below you can see to what level specific models can be selected and what settings are available.

In Closing…

The power of LLMs are primarily being harnessed via prompt chaining and autonomous agents. These two disciplines are supported by prompt engineering tools to fine-tune, store, share, etc. prompts.

These tools have made it possible for LLM Applications (also known as Generative Apps) to make use of multiple LLMs within one application.

Also with continuous LLM model updates, deprecation and new models being released, rapid and benchmarked testing of LLM responses to prompts are becoming increasingly important.

Considering these factors, ChainForge is unique in the sense that it is a no-code to low-code IDE for rapid LLM response testing by making use of a flow approach.

The flow approach also makes it possible to simulate portions of an existing chain.

I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.

Subscribe to HumanFirst Blog

Get the latest posts delivered right to your inbox