Orchestrating Intelligent Agents with Elsa Workflows

Sipke Schoorstra
16 min readSep 1, 2024

--

Outsource Cognitive Tasks and Deploy a Team of Intelligent Agents to Supercharge Your Business Processes

Introduction

A set of new modules just landed in the preview of Elsa 3.3 called Agents.

With Agents, you can define a set of personas that are specialized in a given cognitive task. These agents can then be added to a workflow from where they will perform their work as a background task and produce output that can then feed into subsequent activities — including other agents.

In this context, an agent refers to a preconfigured large language model (LLM) prompt, combined with execution settings and inputs that are provided to the LLM for processing.

The ability to outsource cognitive tasks to intelligent agents enables powerful use cases. For example, if you’re automating content publishing pipelines, you could deploy a Proofreader agent that checks articles for typos, grammar errors and spelling mistakes and fix them. Another agent might create some accompanying artwork, while another agent prepares meta description + keywords for SEO optimization and another agent prepares social media outings.

The potential is virtually limitless.

Setting Up Agents

Let’s explore the Agents module in practice within the context of an actual Elsa Server application.

In case you don’t have an Elsa Server project setup yet, please follow this guide.

Setting up your Elsa Server with support for Agents involves these main steps:

  1. Adding references to the relevant Elsa.Agents.* packages.
  2. Configuring Agents in appsettings.json.
  3. Updating Program.cs .

Package References

Add the following packages to your Elsa Server project:

dotnet add package Elsa.Agents.Core --source https://f.feedz.io/elsa-workflows/elsa-3/nuget/index.json
dotnet add package Elsa.Agents.Activities --source https://f.feedz.io/elsa-workflows/elsa-3/nuget/index.json

These packages contain the core entities and services to define agents and interact with Semantic Kernel.

The Elsa.Agents.Activities package dynamically generates activities based on agents defined in the application. This effectively enables you to execute agents from a workflow, as we’ll see in a moment.

Appsettings.json

Although not the only way, an easy way to define agents is to leverage .NET’s configuration’s system and define agents from appsettings.json.

We’ll define a new parent section called “Agents”, which in turn contains 3 child sections:

  1. “ApiKeys”
  2. “Services”
  3. “Agents”

The following is a complete example of the “Agents” parent section in an appsettings.json file ((save the actual API key value, which you must obtain yourself and is best provided as an Environment variable):

{
"Agents": {
"ApiKeys": [
{
"Name": "MyApiKey",
"Value": "my-api-key"
}
],
"Services": [
{
"Name": "ChatCompletion",
"Type": "OpenAIChatCompletion",
"Settings": {
"ModelId": "gpt-4o",
"ApiKeyRef": "MyApiKey"
}
},
{
"Name": "TextToImage",
"Type": "OpenAITextToImage",
"Settings": {
"ModelId": "dall-e-3",
"ApiKeyRef": "MyApiKey"
}
}
],
"Agents": [
{
"Name": "Copywriter",
"Description": "A persona that writes articles on any topic.",
"Services": [
"ChatCompletion"
],
"FunctionName": "WriteArticle",
"PromptTemplate": "You are a writer of articles about any topic and any additional instructions given to you. The syntax of the article content will be markdown, and your response will be in JSON format containing a \"title\" field and a \"markdown\" field using the following JSON structure: { \"title\": \"The title of the Article\", \"markdown\": \"The article content in markdown syntax.\" }. This is the topic: {{$Topic}}. The article consists of around {{$ParagraphCount}} paragraphs. The article must be written in the language {{$Language}} using the following style: {{$WritingStyle}}. Additional instructions (if any): {{$AdditionalInstructions}}.",
"InputVariables": [
{
"Name": "Topic",
"Description": "The topic to write an article about.",
"Type": "string"
},
{
"Name": "ParagraphCount",
"Description": "The number of paragraphs to write.",
"Type": "int"
},
{
"Name": "Language",
"Description": "The language to write the text in.",
"Type": "string"
},
{
"Name": "WritingStyle",
"Description": "The style to write the text in.",
"Type": "string"
},
{
"Name": "AdditionalInstructions",
"Description": "Any additional instructions.",
"Type": "string"
}
],
"OutputVariable": {
"Name": "Article",
"Description": "The article consisting of a title and markdown text."
},
"ExecutionSettings": {
"MaxTokens": null,
"Temperature": 0.5,
"TopP": 0.9,
"PresencePenalty": 0.5,
"FrequencyPenalty": 0.5,
"ResponseFormat": "json_object"
}
}
]
}
}

In the above example, we defined one agent called “Copywriter” and configured it to act as an agent that is skilled in writing articles in a given language on a given topic.

Notice that we gave this agent the following prompt:

You are a writer of articles about any topic and any additional instructions
given to you.
The syntax of the article content will be markdown, and your response will
be in JSON format containing a "title" field and a "markdown" field using
the following JSON structure:

{
"title": "The title of the Article",
"markdown": "The article content in markdown syntax."
}.

This is the topic: {{$Topic}}.

The article consists of around {{$ParagraphCount}} paragraphs.

The article must be written in the language {{$Language}} using the
following style: {{$WritingStyle}}.

Additional instructions (if any): {{$AdditionalInstructions}}.

The placeholders embedded in the double curly braces ({{$SomePlaceholder}}) will be substituted with actual values we provide via input variables.

The input variables are defined in the “InputVariables” array and contain metadata that will be leveraged by the Elsa.Agents.Activities module to generate activity input descriptors.

Program.cs

Update your Program.cs file as follows:

builder.Services.AddElsa(elsa =>
{
// Enabling the Agent Activities feature will automatically enable the core Agent feature, since that's dependency feature.
elsa.UseAgentActivities();
});

// Bind `AgentsOptions` against the "Agents" configuration section.
builder.Services.Configure<AgentsOptions>(options =>
{
builder.Configuration.GetSection("Agents").Bind(options);
});

Executing Agents from Workflows

With that in place, start your Elsa Server and Elsa Studio, and go ahead and create a new workflow. You will see a new Activity category called “Agents”:

A new section “Agents” appears when there’s at least one Agent defined.

Now, as you define additional agents in appsettings.json, additional activities will appear here.

Let’s add an instance of the Copywriter activity to the workflow and configure it. For example:

In order to capture the output of the activity, define a Workflow Variable and bind it to the output.

Create a new variable called “Article” of type “object”.
Bind the Article variable to the Copywriter’s output.

With that in place, it’s time to hit that Play button:

Try the workflow by clicking the Play button.

After a few seconds, we’ll see the workflow completing:

A job well done!

Elsa Studio + Agents UI

Although defining agents using configuration might work well in some cases, it would be much more powerful if we could quickly define new agents on the fly without needing to restart our application. Additionally, future improvements might include test-driving agents right from the UI.

In case you don’t have an Elsa Studio project setup, please go ahead and follow this guide.

To do so, we need to do two things:

  1. Update Elsa Server with packages that provide storage for agent definitions and exposes a REST API.
  2. Update Elsa Studio with a package that installs the necesary UI components.

Update Elsa Server

Install the following packages:

dotnet add package Elsa.Agents.Api --source https://my.custom.feed/v3/index.json
dotnet add package Elsa.Agents.Persistence.EntityFrameworkCore.Sqlite --source https://my.custom.feed/v3/index.json

Update Program.cs as follows:

elsa
.UseAgentActivities()
.UseAgentPersistence(persistence => persistence.UseEntityFrameworkCore(ef => ef.UseSqlite(sqliteConnectionString)))
.UseAgentsApi();

Restart the application.

Update Elsa Studio

Add the following packages:

dotnet add package Elsa.Studio.Agents --source https://my.custom.feed/v3/index.json

Update Program.cs as follows:

builder.Services.AddAgentsModule(backendApiConfig);

In case you’re wondering about that backendApiConfig variable, it is defined as follows:

var backendApiConfig = new BackendApiConfig
{
ConfigureBackendOptions = options => configuration.GetSection("Backend").Bind(options),
ConfigureHttpClientBuilder = options => options.AuthenticationHandler = typeof(AuthenticatingApiHttpMessageHandler),
};

Restart the Elsa Studio project and refresh the browser. You should now see two new menu item sections:

Two new menu item sections appear: General/Agents and Settings/Agents.

The first things we should do now are:

  1. Register our API key.
  2. Define Services.

Register API Key

To register an API key, navigate to the API Keys page and click the Create API Key button.

From there, simply enter a name and the actual API key.

Register your API Key.

Next, define two new services:

Register two services: ChatCompletion and TextToImage.

Notice that both services reference the API Key by name that we created in the previous step.

Orchestrating Agents

Now that we have done our job, it is time to spawn a team of agents and put them to work.

Let’s create the following agents:

  • Copywriter
  • SEO Specialist
  • Translator
  • Artist
  • Social Media Specialist

Each of these are specialized personas that we will orchestrate in a workflow to perform the following tasks:

  • Write a piece of content on a given topic in a given language.
  • Extract keywords and generate a description for SEO purposes (think meta description + keywords).
  • Translate the article in two more languages.
  • Create artwork to accompany the article.
  • Write a catchy message that can be shared on all of the socials.

Creating Agents

To create a new agent via the UI, simply navigate to the Agents screen and create a new one. For each agent, we can specify many settings, which I will supply here in JSON format.

The module doesn’t currently support importing/exporting JSON, so unfortunately, you will have to copy and paste individual fields by hand for now.

Copywriter

The Copywriter is responsible for writing articles. Here are its settings:

{
"Name": "Copywriter",
"Description": "A persona that writes articles on any topic.",
"Services": [
"ChatCompletion"
],
"FunctionName": "WriteArticle",
"PromptTemplate": "You are a writer of articles about any topic and any additional instructions given to you. The syntax of the article content will be markdown, and your response will be in JSON format containing a \"title\" field and a \"markdown\" field using the following JSON structure: { \"title\": \"The title of the Article\", \"markdown\": \"The article content in markdown syntax.\" }. This is the topic: {{$Topic}}. The article consists of around {{$ParagraphCount}} paragraphs. The article must be written in the language {{$Language}} using the following style: {{$WritingStyle}}. Additional instructions (if any): {{$AdditionalInstructions}}.",
"InputVariables": [
{
"Name": "Topic",
"Description": "The topic to write an article about.",
"Type": "string"
},
{
"Name": "ParagraphCount",
"Description": "The number of paragraphs to write.",
"Type": "int"
},
{
"Name": "Language",
"Description": "The language to write the text in.",
"Type": "string"
},
{
"Name": "WritingStyle",
"Description": "The style to write the text in.",
"Type": "string"
},
{
"Name": "AdditionalInstructions",
"Description": "Any additional instructions.",
"Type": "string"
}
],
"OutputVariable": {
"Name": "Article",
"Description": "The article consisting of a title and markdown text."
},
"ExecutionSettings": {
"MaxTokens": null,
"Temperature": 0.5,
"TopP": 0.9,
"PresencePenalty": 0.7,
"FrequencyPenalty": 0.7,
"ResponseFormat": "json_object"
}
}

SEO Specialist

The SEO Specialist will do two things: extract keywords from a given text, and write a short description about the text.

{
"Name": "SeoSpecialist",
"Description": "A persona that extracts relevant keywords and creates a meta description, optimized for search engines.",
"Services": [
"ChatCompletion"
],
"FunctionName": "ExtractKeywordsAndMetaDescription",
"PromptTemplate": "You are an SEO Specialist. Extract relevant keywords and create a meta description that is optimized for search engines from the following text: {{$Content}}. Your response must be using the following JSON format: { \"keywords\": [\"keyword1\", \"keyword2\"], \"description\": \"The meta description\" }.",
"InputVariables": [
{
"Name": "Content",
"Description": "The content to extract from.",
"Type": "string"
}
],
"OutputVariable": {
"Name": "SeoResult",
"Description": "The description and keywords.",
"Type": "object"
},
"ExecutionSettings": {
"Temperature": 0.5,
"TopP": 0.9,
"PresencePenalty": 0.5,
"FrequencyPenalty": 0.5
}
}

Translator

The Translator can translate any given text to any given language that the configured model supports. Given that our Copywriter produces a JSON object consisting of a Title and a Markdown field, we’ll instruct the Translator to take any JSON input and simply translate the contents of each field.

{
"Name": "Translator",
"Description": "A persona that translates text into a target language.",
"Services": [
"ChatCompletion"
],
"FunctionName": "Translate",
"PromptTemplate": "You are a Translator. Translate the provided text to the target language using the specified language or culture code. Do not change the content or meaning of the text. The text will be provided in the form of a JSON object, where each field contains text to be translated. Your response will be in the exact same JSON format, but with the original text values replaced with the translation for each field. The JSON containing the fields to translate: {{$Content}}. The target language or culture code: {{$Language}}",
"InputVariables": [
{
"Name": "Content",
"Description": "The content to translate.",
"Type": "string"
},
{
"Name": "Language",
"Description": "The language or culture code to translate the content to.",
"Type": "string"
}
],
"OutputVariable": {
"Name": "Content",
"Description": "The translation result.",
"Type": "object"
},
"ExecutionSettings": {
"MaxTokens": null,
"Temperature": 0.2,
"TopP": 0.9,
"PresencePenalty": 0.1,
"FrequencyPenalty": 0.1,
"ResponseFormat": "json_object"
}
}

Artist

The Artist can create corresponding artwork for any article.

{
"Name": "Artist",
"Description": "A persona that creates an image that visualizes the essence of a given text.",
"Services": [
"ChatCompletion",
"TextToImage"
],
"FunctionName": "DesignGraphics",
"PromptTemplate": "You are an Artist that generates an image by interpreting the essence of the following text: {{$Content}}. Your response will be JSON in the following format: {\"imageUrl\": \"https://dalle.com/the-image.png\"}.",
"InputVariables": [
{
"Name": "Content",
"Description": "The content to design graphics for.",
"Type": "String"
}
],
"OutputVariable": {
"Name": "Image",
"Description": "An URL to the created artwork.",
"Type": "object"
},
"ExecutionSettings": {
"MaxTokens": null,
"Temperature": 1,
"TopP": 0.9,
"PresencePenalty": 0.0,
"FrequencyPenalty": 0.0,
"ResponseFormat": "json_object"
},
"Plugins": [
"ImageGenerator"
]
}

Social Media Specialist

The Social Media Specialist can write catchy posts for sharing on social media.

{
"Name": "Social Media Specialist",
"Description": "A persona that specializes in writing catchy social media posts based on a given piece of content.",
"Services": [
"ChatCompletion"
],
"FunctionName": "WriteSocialMediaPost",
"PromptTemplate": "You are a Social Media Specialist. Write a catchy social media post based on the following text: {{$Content}}. The post should be optimized for social media platforms such as X (formerly Twitter) , Facebook, LinkedIn, and Instagram. Your response must be in JSON format and contain a single field named 'post' with the text of the social media post. Example: { \"post\": \"Some catchy social media post.\" }",
"InputVariables": [
{
"Name": "Content",
"Description": "The content to write a social media post for.",
"Type": "string"
}
],
"OutputVariable": {
"Name": "Post",
"Description": "The generated social media post.",
"Type": "object"
},
"ExecutionSettings": {
"Temperature": 0.9,
"TopP": 0.9,
"PresencePenalty": 0.7,
"FrequencyPenalty": 0.7
}
}

The final list should look like this:

Our new army of content specialists.

Creating the Workflow

Now it’s time to put our army of content specialist to work.

Create a new workflow called “Create and Publish Content” and go ahead and drag and drop our agents as you see fit. For example:

A simple, yet powerful content processing workflow.

As you might have noticed, the majority of these agents will run in parallel. This is possible due to the fact that each agent activity’s Kind is configured to be a Job.

Elsa’s workflow engine offers a robust feature where activities designated as ‘Job’ are executed asynchronously as background jobs. This is particularly useful when you have multiple jobs queued up; they are processed concurrently, in parallel. This parallel execution capability ensures that tasks are handled efficiently, maximizing throughput and minimizing response time.

Running this workflow will produce content that is similar to the following:

Copywriter:

{
"title":"The Power of Microsoft Semantic Kernel",
"markdown":"# The Power of Microsoft Semantic Kernel\n\n## Unleashing the Force of Data Understanding\n\nIn a galaxy not so far away, Microsoft has harnessed the power of the Semantic Kernel, a tool that brings unparalleled clarity to the chaotic universe of data. Like a Jedi wielding a lightsaber, this technology slices through ambiguity and illuminates the path to insight. With its ability to understand context and meaning, the Semantic Kernel transforms raw data into actionable intelligence.\n\n## The Architecture: A Masterpiece in Design\n\nCrafted with precision akin to a lightsaber\u0027s hilt, the architecture of Microsoft\u0027s Semantic Kernel is both elegant and formidable. It integrates seamlessly with existing systems, leveraging machine learning algorithms that are as precise as a Jedi\u0027s mind trick. This architectural marvel ensures that businesses can tap into their data reserves with unprecedented ease and accuracy.\n\n## Real-World Applications: Harnessing Galactic Potential\n\nImagine navigating through vast amounts of unstructured data like a starship through hyperspace. The Semantic Kernel makes this possible by enabling real-time analysis and decision-making. From enhancing customer experiences to optimizing supply chains, its applications are as diverse as the stars in the sky. Businesses across sectors are already feeling the force of this revolutionary technology.\n\n## Empowering Users: The Jedi Council of Data Science\n\nMicrosoft\u0027s vision extends beyond mere technology; it empowers users to become masters in their own right. With intuitive interfaces and robust support systems, even those new to data science can harness the full potential of the Semantic Kernel. It\u0027s like being trained by Yoda himself—users gain wisdom and skill rapidly, transforming them into powerful guardians of their organization\u0027s knowledge.\n\n## The Future: A New Hope for Data Management\n\nAs we look towards the horizon, it\u0027s clear that Microsoft\u0027s Semantic Kernel represents a new hope for data management. Its capabilities will continue to evolve, bringing even greater precision and understanding to complex datasets. In this ever-expanding universe of information, having such a powerful ally ensures that organizations remain ahead in their quest for knowledge and innovation.\n"
}

SEO Specialist:

{
"keywords":["Microsoft Semantic Kernel","data understanding","machine learning","real-time analysis","data management"],
"description":"Discover the power of Microsoft\u0027s Semantic Kernel, a revolutionary tool that transforms raw data into actionable intelligence with unparalleled clarity and precision. Learn how its advanced architecture and real-time analysis capabilities empower businesses to optimize operations and enhance customer experiences."
}

Social Media Specialist:

{
"post":"\uD83C\uDF0C Unleash the power of Microsoft Semantic Kernel! \uD83D\uDE80 Like a Jedi\u0027s lightsaber, it cuts through data chaos, turning raw info into actionable insights. With precision architecture and real-time analysis, it\u0027s transforming businesses across galaxies. Ready to become a Data Science Jedi? #Microsoft #DataScience #Innovation"
}

Translator (French):

{
"title":"La Puissance du Noyau Sémantique de Microsoft",
"markdown":"# La Puissance du Noyau Sémantique de Microsoft\n\n## Libérer la Force de la Compréhension des Données\n\nDans une galaxie pas si lointaine, Microsoft a exploité la puissance du Noyau Sémantique, un outil qui apporte une clarté inégalée à l\u0027univers chaotique des données. Comme un Jedi maniant un sabre laser, cette technologie tranche à travers l\u0027ambiguïté et illumine le chemin vers la perspicacité. Avec sa capacité à comprendre le contexte et le sens, le Noyau Sémantique transforme les données brutes en intelligence exploitable.\n\n## L\u0027Architecture : Un Chef-d\u0027œuvre de Conception\n\nConçu avec une précision semblable à celle d\u0027un manche de sabre laser, l\u0027architecture du Noyau Sémantique de Microsoft est à la fois élégante et redoutable. Elle s\u0027intègre parfaitement aux systèmes existants, tirant parti d\u0027algorithmes d\u0027apprentissage automatique aussi précis qu\u0027un tour de passe-passe Jedi. Cette merveille architecturale garantit que les entreprises peuvent exploiter leurs réserves de données avec une facilité et une précision sans précédent.\n\n## Applications Réelles : Exploiter un Potentiel Galactique\n\nImaginez naviguer à travers d\u0027immenses quantités de données non structurées comme un vaisseau spatial dans l\u0027hyperespace. Le Noyau Sémantique rend cela possible en permettant une analyse et une prise de décision en temps réel. De l\u0027amélioration des expériences client à l\u0027optimisation des chaînes d\u0027approvisionnement, ses applications sont aussi diverses que les étoiles dans le ciel. Les entreprises de tous les secteurs ressentent déjà la force de cette technologie révolutionnaire.\n\n## Autonomiser les Utilisateurs : Le Conseil Jedi de la Science des Données\n\nLa vision de Microsoft va au-delà de la simple technologie ; elle permet aux utilisateurs de devenir maîtres en leur propre droit. Avec des interfaces intuitives et des systèmes de support robustes, même ceux qui sont nouveaux dans la science des données peuvent exploiter tout le potentiel du Noyau Sémantique. C\u0027est comme être formé par Yoda lui-même—les utilisateurs acquièrent rapidement sagesse et compétence, se transformant en puissants gardiens du savoir de leur organisation.\n\n## L\u0027Avenir : Un Nouvel Espoir pour la Gestion des Données\n\nEn regardant vers l\u0027horizon, il est clair que le Noyau Sémantique de Microsoft représente un nouvel espoir pour la gestion des données. Ses capacités continueront d\u0027évoluer, apportant encore plus de précision et compréhension aux ensembles complexes de données. Dans cet univers en constante expansion d\u0027informations, avoir un allié aussi puissant garantit que les organisations restent en avance dans leur quête pour le savoir et l\u0027innovation.\n"
}

Translator (Dutch):

{
"title":"De Kracht van Microsoft Semantic Kernel",
"markdown":"# De Kracht van Microsoft Semantic Kernel\n\n## De Kracht van Data Begrip Ontketenen\n\nIn een niet zo verre melkweg heeft Microsoft de kracht van de Semantic Kernel benut, een tool die ongeëvenaarde helderheid brengt in het chaotische universum van data. Net als een Jedi met een lichtzwaard snijdt deze technologie door ambiguïteit en verlicht het pad naar inzicht. Met zijn vermogen om context en betekenis te begrijpen, transformeert de Semantic Kernel ruwe data in bruikbare intelligentie.\n\n## De Architectuur: Een Meesterwerk in Ontwerp\n\nMet precisie vervaardigd zoals het handvat van een lichtzwaard, is de architectuur van Microsoft\u0027s Semantic Kernel zowel elegant als formidabel. Het integreert naadloos met bestaande systemen en maakt gebruik van machine learning-algoritmen die net zo precies zijn als de gedachtenkracht van een Jedi. Dit architectonische wonder zorgt ervoor dat bedrijven hun datareserves met ongekende eenvoud en nauwkeurigheid kunnen benutten.\n\n## Toepassingen in de Praktijk: Galactisch Potentieel Benutten\n\nStel je voor dat je door enorme hoeveelheden ongestructureerde data navigeert als een ruimteschip door hyperspace. De Semantic Kernel maakt dit mogelijk door realtime analyse en besluitvorming mogelijk te maken. Van het verbeteren van klantervaringen tot het optimaliseren van toeleveringsketens, de toepassingen zijn net zo divers als de sterren aan de hemel. Bedrijven in verschillende sectoren voelen al de kracht van deze revolutionaire technologie.\n\n## Gebruikers Empoweren: De Jedi Raad van Datawetenschap\n\nMicrosoft\u0027s visie gaat verder dan alleen technologie; het stelt gebruikers in staat om meesters op hun eigen gebied te worden. Met intuïtieve interfaces en robuuste ondersteuningssystemen kunnen zelfs nieuwkomers in datawetenschap het volledige potentieel van de Semantic Kernel benutten. Het is alsof ze getraind worden door Yoda zelf—gebruikers verwerven snel wijsheid en vaardigheden, waardoor ze krachtige bewakers worden van de kennis binnen hun organisatie.\n\n## De Toekomst: Een Nieuwe Hoop voor Databeheer\n\nAls we naar de horizon kijken, is het duidelijk dat Microsoft\u0027s Semantic Kernel een nieuwe hoop vertegenwoordigt voor databeheer. Zijn capaciteiten zullen blijven evolueren, waardoor nog grotere precisie en begrip mogelijk wordt bij complexe datasets. In dit steeds uitdijende universum van informatie zorgt zo\u0027n krachtige bondgenoot ervoor dat organisaties voorop blijven lopen in hun zoektocht naar kennis en innovatie.\n"
}

Artist:

Generated by the Artist agent

Conclusion

In this article, I introduced the new Agents module for Elsa Workflows. We went over setting up the module in Elsa Server and Elsa Studio and then created a small set of agents and a workflow to exercise the agents.

Combining workflows with AI offers a reliable, predictable and repeatable process in a powerful manner.

Resources

--

--