Transforming user data entry: implementing Medplum Bots for Questionnaire automation

Questionnaires and their corresponding QuestionnaireResponses are fundamental resources in any FHIR-compliant EHR system. They serve as primary entry points for user-provided data and, as developers, we interact with them in almost all form implementations. When handling forms, QuestionnaireResponses function as intermediary resources that will later be processed into specialized ones such as Patients, Observations, Conditions, and many others.

But how do we efficiently implement this? Organizing a workflow without a proper structure in place can be a daunting task, and that’s how Medplum Bots can help us the most: by providing tools and automations to facilitate this process.

In this post, we’ll be exploring what Medplum Bots are, why it’s best to use them instead of implementing separate custom backends, how we can use Bots to automate our Questionnaires workflow, and how to easily integrate them with our Frontend.

What are Medplum Bots?

Bots are Medplum’s implementation of serverless functions. The functions run on AWS Lambda by default, exempting us from configuring anything other than how the Bot is called and the Bot’s code itself, while Medplum handles the deploy process. Within a Bot, we have access to any resource inside the Medplum project, as well as access to all functions from Medplum’s SDK. Bots are, essentially, Medplum’s custom backend approach. 

However, they do not have access to custom frontend code. To help with code organization (and avoid repetition), Medplum provides support to Lambda Layers, which allows us to have separate files to handle reusable code that’ll be deployed alongside the Bots.

Keep in mind that Bots must be activated on your Medplum project before they’re made available to use. For more information, read the Bots documentation page. Checkout Medplum’s Pricing page as well, especially the row regarding Bot invocation limits.

Struggling with Medplum?
Our experienced team is ready to step in.

Why use Bots?

Since Medplum’s SDK is tailored to be used by a frontend application, it is possible to write any operation directly into the frontend code, for both the creation of a QuestionnaireResponse and the conversion into specialized resources. This approach is not ideal as it requires the developer to handle extensive data processing directly in the frontend, which might result in assigning specific permission levels to the user that wouldn’t be required otherwise, as well as negatively impacting the application’s performance.

We can see from the image above that we are relying entirely on the frontend side to send all processed data to the Medplum server. By using a Bot, we are able to delegate all of the processing work to it, freeing the frontend from handling these more sensitive, heavier, tasks. 

As demonstrated above, we can link a Bot to a Questionnaire so it triggers whenever a related QuestionnaireResponse is created, updated, or deleted. This is one of many ways Medplum provides to trigger a Bot, all of which can be found on Medplum’s documentation.

Creating a Bot on Medplum

The Bot creation process is simple and doesn’t require much configuration. It can be done directly through Medplum’s admin interface by going to the Bots page then clicking on New. This will open the “Create new Bot” page where we’ll be able to give the Bot a name, description, and an Access Policy, though only the name is required:

After this, we are redirected to the new Bot’s instance page containing the Editor tab. This tab provides an interface with the Bot’s code as well as a field for parameters needed for its execution, and an area to log its output.

Once you have the code ready, click on Save, then on Deploy to apply the changes on Medplum’s server and deploy to AWS Lambda. This page lets us Execute a Bot at any time after deploying it, which is great for development and debugging purposes.

However, we don’t want to have to deploy Bots manually in production. For a proper deploy pipeline, we can use Medplum’s deploy endpoint.

Questionnaire and QuestionnaireResponse

The first step to trigger a Bot when a QuestionnaireResponse is interacted with is to link both to a Questionnaire resource. Medplum provides a seamless interface that links a Bot to a Questionnaire, which then triggers whenever a related QuestionnaireResponse is interacted with.

Behind the scenes, Medplum is actually creating a Subscription resource responsible for triggering the Bot whenever an event occurs. This whole process is just a shortcut following a good practice when it comes to Questionnaire interactions, seamlessly integrated with Medplum. In the image above, the event is the creation of a QuestionnaireResponse related to the “patient-intake” Questionnaire. Medplum has this process well documented. You should also be able to find the Subscription saved in your Medplum project.

Why use them?

Besides having a built-in integration with Bots, why should we use Questionnaire and QuestionnaireResponse instead of creating all specialized resources during a form’s submit action? By using them, we are:

• Properly following FHIR’s structure and good practices, which makes Medplum easier to use as it’s made with FHIR interoperability in mind;
• Check out the QuestionnaireForm and QuestionnaireBuilder components for specific Questionnaire and QuestionnaireResponse use cases;
• Separating form logic from specialized resources. Doing so lets us choose when they are created, as we get to decide when the Bot should be triggered;
• Keeping a record of what was answered by the user at a specific time, in a specific Questionnaire;. This can provide a Practitioner with better audit capabilities, as well as retrigger a Bot in case of a previous failure;
• Improving frontend performance, as mentioned earlier. We are moving the heavy operations to a backend, rather than executing them on frontend.

Practical Example: Patient Intake

For this example, we’ll be exploring Medplum’s own Patient Intake demo. The intake process is usually the first contact a patient will have with your EHR, and it’s also where we’ll be collecting the most information from them. This makes it likely to be one of the biggest Questionnaires in the system.

‍After setting up the demo, we can go to the New Patient form by clicking on Medplum’s logo to open the sidebar, then clicking on New Patient:

In the form, all questions are being rendered by QuestionnaireForm, which is reading the Questionnaire resource and rendering it to follow Medplum’s specifications. Take the Allergies section, for instance:

This section is being built as described in the Questionnaire resource, formatted in JSON:

{
  "linkId": "allergies",
  "text": "Allergies",
  "type": "group",
  "repeats": true,
  "item": [
    {
      "linkId": "allergy-substance",
      "text": "Substance",
      "type": "choice",
      "answerValueSet": "http://cts.nlm.nih.gov/fhir/ValueSet/2.16.840.1.113762.1.4.1186.8"
    },
    {
      "linkId": "allergy-reaction",
      "text": "Reaction",
      "type": "string"
    },
    {
      "linkId": "allergy-onset",
      "text": "Onset",
      "type": "dateTime"
    }
  ]
}

‍

By using a Questionnaire, we get to correlate responses by their linkId when creating a related QuestionnaireResponse. This will help us later when building the Bot.

Triggering the Bot

The Intake form will collect data to be saved in the QuestionnaireResponse, formatted similarly to the related Questionnaire. As an example, consider the following answers:

This will generate a QuestionnaireResponse, containing the answers we’ve filled in the form:

[..., {
  "id": "id-94",
  "linkId": "allergies",
  "text": "Allergies",
  "item": [
    {
      "id": "id-95",
      "linkId": "allergy-substance",
      "text": "Substance",
      "answer": [
        {
          "valueCoding": {
            "system": "http://snomed.info/sct",
            "code": "102264005",
            "display": "Cheese (substance)"
          }
        }
      ]
    },
    {
      "id": "id-96",
      "linkId": "allergy-reaction",
      "text": "Reaction",
      "answer": [
        {
          "valueString": "Mild"
        }
      ]
    },
    {
      "id": "id-97",
      "linkId": "allergy-onset",
      "text": "Onset",
      "answer": [
        {
          "valueDateTime": "2024-12-20T03:00:00.000Z"
        }
      ]
    }
  ]
},
{
  "id": "id-147",
  "linkId": "allergies",
  "text": "Allergies",
  "item": [
    {
      "id": "id-148",
      "linkId": "allergy-substance",
      "text": "Substance",
      "answer": [
        {
          "valueCoding": {
            "system": "http://snomed.info/sct",
            "code": "102263004",
            "display": "Eggs"
          }
        }
      ]
    },
    {
      "id": "id-149",
      "linkId": "allergy-reaction",
      "text": "Reaction",
      "answer": [
        {
          "valueString": "Severe"
        }
      ]
    },
    {
      "id": "id-150",
      "linkId": "allergy-onset",
      "text": "Onset",
      "answer": [
        {
          "valueDateTime": "2024-12-20T03:00:00.000Z"
        }
      ]
    }
  ]
},
{
  "id": "id-98",
  "linkId": "medications",
  "text": "Current medications",
  "item": [
    {
      "id": "id-99",
      "linkId": "medication-code",
      "text": "Medication Name",
      "answer": [
        {
          "valueCoding": {
            "system": "http://www.nlm.nih.gov/research/umls/rxnorm",
            "code": "1007388",
            "display": "Lactase / rennet"
          }
        }
      ]
    }
  ]
}, ...]

‍

These answers can be mapped to two different resources: AllergyIntolerance and MedicationRequest, but we won’t be creating them in the frontend. Instead, the creation of the QuestionnaireResponse will trigger the Bot, which will then read the answers and create the necessary specialized resources. 

const response: QuestionnaireResponse = event.input;

When triggered, the Bot will receive the entire QuestionnaireResponse as an argument. As seen in the code above, the first step is to retrieve it from the event’s input. Once we have access to the user’s answers, we can parse it however we need.

const questionnaire: Questionnaire = await medplum.readReference({
  reference: response.questionnaire,
});

const allergies = getGroupRepeatedAnswers(
  questionnaire, response, "allergies",
);
for (const allergy of allergies) {
  const code = allergy['allergy-substance']?.valueCoding;

  if (!code) {
    continue;
  }

  const reaction = allergy['allergy-reaction']?.valueString;
  const onsetDateTime = allergy['allergy-onset']?.valueDateTime;

  await medplum.upsertResource(
    {
      resourceType: 'AllergyIntolerance',
      meta: {
        profile: [PROFILE_URLS.AllergyIntolerance],
      },
      clinicalStatus: {
        text: 'Active',
        coding: [
          {
            system: 'http://hl7.org/fhir/ValueSet/allergyintolerance-clinical',
            code: 'active',
            display: 'Active',
          },
        ],
      },
      verificationStatus: {
        text: 'Unconfirmed',
        coding: [
          {
            system: 'http://hl7.org/fhir/ValueSet/allergyintolerance-verification',
            code: 'unconfirmed',
            display: 'Unconfirmed',
          },
        ],
      },
      patient: createReference(patient),
      code: { coding: [code] },
      reaction: reaction ? [{ manifestation: [{ text: reaction }] }] : undefined,
      onsetDateTime,
    },
    {
      patient: getReferenceString(patient),
      code: `${code.system}|${code.code}`,
    }
  );
}

The code above handles the creation of AllergyIntolerance resources. They are created based on answers from the Allergies section of the QuestionnaireResponse. To accomplish that, we are:

• Collecting all allergy-related answers using getGroupRepeatedAnswers;
• Looping through each allergy and accessing its values by linkId, as defined in the QuestionnaireResponse;
• Formatting the data to fit the AllergyIntolerance FHIR format, using the answers to fill it as necessary;

Here we are doing the same steps for MedicationRequest, but with the Medications section instead. The full Bot code can be found here, and its utils layer file here. All helpers are either coming from Medplum’s SDK, or the utils layer. 

Once the execution is done, the specialized resources should have been created.

Auditing Bot executions

Just like any other code, Bots can fail. In order to give us visibility of when that happens, Medplum registers every Bot execution in the AuditEvent resource. The events are made available directly in the Bot’s page on Medplum’s admin interface.

In the example above, we have 2 different executions:

• In the bottom one, the Bot failed due to a bug in the code. Here we can read the traceback that led to the error and use it to investigate what happened;
• At the top, we have the execution that succeeded. This won’t provide a traceback, but we can identify it as a success by the number in the Outcome column: 0.

Conclusion

When working with Bots, Medplum offers a strong developer-friendly experience that facilitates automation while providing tools for deploying, debugging and properly auditing them. This allowed us to spend less time on environment setup and more on delivering impactful features. 

Paired with Questionnaire, we minimized the risk of data loss with forms that perform nicely and give users a reliable experience while still being FHIR-compliant.

Do you have questions about implementing Bots or Questionnaires in your system? We've worked extensively with these tools and would happily share specific implementation details or discuss any technical challenges you face. We're here to share our expertise and guide you through the process. Reach out to start a conversation about your product automation needs.