Skip to content

Bedrock runtime error when using prompt caching #7100

Closed
@glc-froussel

Description

@glc-froussel

Checkboxes for prior research

Describe the bug

With AWS Lambda, using the ConverseCommand with a cache point causes the following runtime error :

{
  "errorType": "TypeError",
  "errorMessage": "Cannot read properties of undefined (reading '0')",
  "trace": [
    "TypeError: Cannot read properties of undefined (reading '0')",
    "    at Object.visit (/var/runtime/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:717:36)",
    "    at se_SystemContentBlock (/var/runtime/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1904:29)",
    "    at /var/runtime/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1912:12",
    "    at Array.map (<anonymous>)",
    "    at se_SystemContentBlocks (/var/runtime/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1911:41)",
    "    at system (/var/runtime/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1138:45)",
    "    at applyInstruction (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:1115:27)",
    "    at take (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:1083:5)",
    "    at se_ConverseCommand (/var/runtime/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1129:35)",
    "    at /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:71:25"
  ]
}

Without cache point, everything is working fine.

Regression Issue

  • Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/package-name@version, ...

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

NodeJS 22.X

Reproduction Steps

package.json :

  "dependencies": {
    "@aws-sdk/client-bedrock": "^3.816.0",
    "@aws-sdk/client-bedrock-runtime": "^3.816.0",
    "@smithy/node-http-handler": "^4.0.5"
  },
  "devDependencies": {
    "@tsconfig/node22": "^22.0.1",
    "@tsconfig/strictest": "^2.0.5",
    "@types/aws-lambda": "^8.10.149",
    "@types/node": "^22",
    "aws-cdk": "^2.1016.0",
    "aws-cdk-lib": "^2.196.0",
    "constructs": "^10.4.2",
    "esbuild": "^0.25.4",
    "tsx": "^4.19.4",
    "typescript": "^5.8.3"
  }

hello-bedrock-handler.ts :

import { BedrockRuntimeClient, ConverseCommand, ConverseCommandInput } from '@aws-sdk/client-bedrock-runtime';

const client = new BedrockRuntimeClient({
  region: 'us-east-1',
});

export const handler = async (): Promise<any> => {
  const input: ConverseCommandInput = {
    modelId: 'amazon.nova-lite-v1:0',
    system: [
      {
        text: 'Tu es un assistant utile qui aide les utilisateurs à obtenir des informations sur la météo actuelle dans une ville',
      },
      {
        cachePoint: {
          type: 'default',
        },
      },
    ],
    messages: [
      {
        role: 'user',
        content: [
          {
            text: 'Salut, ca va ?',
          },
        ],
      },
    ],
  };

  const response = await client.send(new ConverseCommand(input));

  return response.output;
};

cdk construct :

import { join, resolve } from 'path';

import { Construct } from 'constructs';
import { RemovalPolicy, Duration, Stack, StackProps } from 'aws-cdk-lib';
import { LogGroup, RetentionDays } from 'aws-cdk-lib/aws-logs';
import { NodejsFunction, OutputFormat, SourceMapMode } from 'aws-cdk-lib/aws-lambda-nodejs';
import { Role, ServicePrincipal, ManagedPolicy, PolicyStatement, Effect } from 'aws-cdk-lib/aws-iam';
import { Function, Code, Runtime, Architecture } from 'aws-cdk-lib/aws-lambda';

export class BedrockCachingStack extends Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    super(scope, id, props);

    const app = 'bedrock-caching';
    const env = 'dev';

    const role = new Role(this, 'Role', {
      roleName: `${app}-function-${env}`,
      assumedBy: new ServicePrincipal('lambda.amazonaws.com'),
      managedPolicies: [ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaVPCAccessExecutionRole')],
    });

    role.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        actions: ['bedrock:*'],
        resources: ['*'],
      })
    );

    const functionName = `${app}-hello-${env}`;

    const logGroup = new LogGroup(this, 'LogGroup', {
      logGroupName: `/aws/lambda/${functionName}`,
      retention: RetentionDays.ONE_MONTH,
      removalPolicy: RemovalPolicy.DESTROY,
    });

    const function1 = new NodejsFunction(this, 'Function', {
      runtime: Runtime.NODEJS_22_X,
      memorySize: 512,
      timeout: Duration.seconds(10),
      architecture: Architecture.ARM_64,
      entry: resolve(join(process.cwd(), 'src/handlers', 'hello-bedrock-handler.ts')),
      handler: 'handler',
      functionName,
      logGroup,
      role,
      bundling: {
        platform: 'node',
        format: OutputFormat.CJS,
        forceDockerBundling: false,
        minify: true,
        sourceMap: true,
        sourcesContent: true,
        sourceMapMode: SourceMapMode.INLINE,
        metafile: false,
      },
      awsSdkConnectionReuse: false,
    });

    function1.node.addDependency(logGroup);
  }
}

Then manually trigger the function on AWS Console.

Observed Behavior

With cache point set in the request the error is thrown, without the cache point it's working.

If I execute manually the handler locally (outside Lambda), no error, even with the cache point.

  • This is apparently not a permission error because without the cache point it's working
  • I first though of a packaging issue of the Lambda, but again, without cache point it's working and I tried different packaging setup without luck
  • Behavior happens for nova-lite but also claude 3.5 haiku

Expected Behavior

No error.

Possible Solution

No response

Additional Information/Context

Exactly same error than #6404 (comment)

But the root cause seems different as I am using last versions of aws-sdk.

Metadata

Metadata

Assignees

Labels

bugThis issue is a bug.closed-for-stalenessp3This is a minor priority issue

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions