There might be many cases in which you might want to integrate generative AI capabilities into your application. One way to do this is by leveraging aws bedrock APIs. Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon via a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI.

In this blog post, I will lead you through the process of setting up AWS bedrock and connecting it to a Node.js application for running inferences. I will show you how you can create a simple node js function that does some inference and returns result when it is invoked. I will be using the AWS’s “’Titan Text G1 – Lite’” model for this demonstration.

Prerequisites

Before you begin you need:

• An aws account with a full administrative privileges
• Node.js JavaScript runtime environment
• AWS Command Line Interface (CLI)

Step 1 : Create an IAM User

Our Node.js application will programmatically access AWS Bedrock to execute inferences using the ‘Titan Text G1 – Lite’ model. To achieve this, the application uses an IAM user. In this step we will create that IAM user.

To create an IAM user 

1. Navigate to the Identity Access Management (IAM) Console. 
2. Go to the Users  link in the left side navigation panel
3. Click Create User, and proceed with the outlined steps.

Step 2 : Authorize the user to access the AWS Bedrock service

After creating the user, it is necessary to grant them access to AWS Bedrock. 

To configure permissions

1. In the navigation pane, choose Users.
2. In the list, choose the name of the user you have created earlier.
3. Choose the Permissions tab.
4. Locate the Permissions Policies section, and choose the ‘Add Permissions’ dropdown, followed by clicking the ‘Add Permissions’ button.
5. Select the ‘Attach policies directly’ option. In the Permissions Policies section, search for and choose the AWS-managed policy ‘AmazonBedrockFullAccess,’ click ‘Next,’ review, and confirm the permission addition.

Screenshot of the IAM console’s Add permission page

Step 3 : Generate Access Key

To generate an Access Key for the user

1. Go to the security credentials tab, scroll down to the “Access keys” section 
2. In the Access keys section, choose Create access key.
3. On the Access key best practices & alternatives page, choose “Command Line Interface (CLI)” tick the “I understand the above recommendation and want to proceed to create an access key.” check box and then click Next.
4. On the Retrieve access keys page, choose either Show to reveal the value of your user’s secret access key, or Download .csv file. This is your only opportunity to save your secret access key. After you’ve saved your secret access key in a secure location, choose Done.

Step 4 : Configure aws bedrock service to enable “’Titan Text G1 – Lite’” model access

Amazon Bedrock users need to request access to models before they are available for use. Model access can be managed only in the Amazon Bedrock console. 

You can add access to a model in Amazon Bedrock with the following steps:

1. Open the Amazon Bedrock console at Amazon Bedrock console
2. Go to the Model access link in the left side navigation panel in Amazon Bedrock, or go to the Edit model access page.
3. Select the check box next to the ‘Titan Text G1 – Lite’ model. 
4. Select the Save Changes button in the lower right corner of the page. It may take several minutes to save changes to the Model access page.

Screenshot of aws bedrock console’s request model access page.

Step 5 : Initialize Node.js Application and Install the necessary packages

Now that we have all aws config setups completed we can now create our node js app. This step assumes you have node js installed and running.

1. Open Windows Command Prompt
2. CD into you chosen project directory
3. Initialize npm using the npm init –yes command
4. Install the @aws-sdk/client-bedrock-runtime package. 

Note: The @aws-sdk/client-bedrock-runtime is an AWS SDK for JavaScript

BedrockRuntime Client for Node.js, Browser and React Native. It describes the

API operations for running inference using Bedrock models.

Step 6 : Set up authentication details

At this point, we are setting up our application to interact with AWS services. To achieve this, we’ll be configuring our AWS Command Line Interface (CLI) with the credentials of the user we’ve already created. Assuming you have the AWS CLI installed, we prefer configuring the user credentials through the CLI rather than hard-coding them into our code. This approach enhances security, as storing sensitive credentials directly in the code is not recommended. Following the AWS credentials provider chain, the AWS SDK will automatically search and use the credential within the AWS CLI, ensuring that our application can securely access AWS bedrock.

1. Open the command prompt and initiate the user configuration by entering the command “aws configure.”
2. Enter the Access Key ID.
3. Provide the Secret Access Key.
4. Specify the Region.

Screenshot of windows command prompt window showing the process of aws user configuration

Step 7 : Write a node js code that does inference

In this final step, I will demonstrate how you can make use of the @aws-sdk/client-bedrock-runtime package to interact with AWS Bedrock and send basic requests.

1. Create a new file named index.js.
2. Paste the provided code into the index.js file.

import {

  BedrockRuntimeClient,

  InvokeModelCommand,

} from "@aws-sdk/client-bedrock-runtime"; // ES Modules import

const client = new BedrockRuntimeClient({

  region: "YOUR REGION",

});

const askAi = async (message) => {

  const request = {

    inputText: message,

    textGenerationConfig: {

      temperature: 0.5,

      topP: 1,

      maxTokenCount: 512,

    },

  };

  const input = {

    body: JSON.stringify(request),

    contentType: "application/json",

    accept: "application/json",

    modelId: "amazon.titan-text-lite-v1",

  };

  const command = new InvokeModelCommand(input);

  const response = await client.send(command);

  return JSON.parse(Buffer.from(response.body).toString("utf8")).results[0]

    .outputText;

};

console.log(await askAi("Hi")); //returns "Bot: Hello! How can I assist you today?"

This code snippet is a JavaScript implementation using the AWS SDK for the Bedrock Runtime service. Here’s an explanation of the code:

Import Statements:

import {

  BedrockRuntimeClient,

  InvokeModelCommand,

} from "@aws-sdk/client-bedrock-runtime"; // ES Modules import

These import statements bring in the necessary classes from the @aws-sdk/client-bedrock-runtime module. BedrockRuntimeClient is used for interacting with the Bedrock Runtime service, and InvokeModelCommand represents the command to invoke a model.

Creating BedrockRuntimeClient:

const client = new BedrockRuntimeClient({

  region: "YOUR REGION",

});

An instance of the BedrockRuntimeClient is created with the specified AWS region.

askAi Function:

const askAi = async (message) => {

//...

}

This function, named askAi, is defined as an asynchronous function that takes a message parameter.

Request Configuration:

const request = {

    inputText: message,

    textGenerationConfig: {

      temperature: 0.5,

      topP: 1,

      maxTokenCount: 512,

    },

  };

A request object is created with the inputText set to the provided message, and additional text generation configuration parameters like temperature, topP, and maxTokenCount are specified.

Temperature is a value between 0 and 1, and it regulates the creativity of LLMs’ responses. Use lower temperature if you want more deterministic responses, and use higher temperature if you want more creative or different responses for the same prompt from LLMs on Amazon Bedrock. For all the examples in this prompt guideline, we set temperature = 0.

Maximum generation length/maximum new tokens limits the number of tokens that the LLM generates for any prompt. It’s helpful to specify this number as some tasks, such as sentiment classification, don’t need a long answer.

Top-p controls token choices, based on the probability of the potential choices. If you set Top-p below 1.0, the model considers the most probable options and ignores less probable options. The result is more stable and repetitive completions.

Input Configuration:

const input = {

    body: JSON.stringify(request),

    contentType: "application/json",

    accept: "application/json",

    modelId: "amazon.titan-text-lite-v1",

  };

The input configuration is prepared with the request body as a JSON string, specifying the content type and accept headers as JSON, and setting the modelId to “amazon.titan-text-lite-v1.”

InvokeModelCommand:

const command = new InvokeModelCommand(input);

const response = await client.send(command);

An instance of the InvokeModelCommand is created with the input configuration, and it is sent to the Bedrock Runtime service using the client.send(command) method. The response is then awaited.

Processing Response:

return JSON.parse(Buffer.from(response.body).toString("utf8")).results[0].outputText;

The response body is parsed from a buffer to a UTF-8 encoded string, and the output text from the model is extracted and returned.

Demonstration 

console.log(await askAi("Hi")); //returns "Bot: Hello! How can I assist you today?"

The provided example demonstrates using the AWS Bedrock Runtime service to interact with a language model and generate text based on a given input message (“Hi” in this case). The response “Bot: Hello! How can I assist you today?” is an example of the generated output from the language model.

In summary, this code sets up a client for interacting with the Bedrock Runtime service and defines a function (askAi) that uses this client to invoke a text generation model with a specified input message and configuration. The result is then processed and the output text is returned.

Conclusion

In this blog post, I’ve tried to demonstrate the process of accessing the AWS Bedrock model from your Node.js application and performing inferences. By following the outlined steps, you can establish and incorporate AI functionality seamlessly into your Node.js application. This will allow you to incorporate AI capabilities into your application and leverage the advantages offered by AI functionalities.