core

Create messages for language models like Claude and OpenAI GPTs.

from IPython.display import Image, display
from pathlib import Path

from pathlib import Path

API Exploration

Anthropic’s Claude and OpenAI’s GPT models are some of the most popular LLMs.

Let’s take a look at their APIs and to learn how we should structure our messages for a simple text chat.

openai

from openai import OpenAI

client = OpenAI()

client.responses.create(
  model="gpt-4.1",
  input=[ {"role": "user", "content": "Hello, world!"} ]
)

Hello, world! 👋 How can I help you today?

• id: resp_68a4d422397c81a19fb6b4d899b25e7b029dfdae1caafc84
• created_at: 1755632674.0
• error: None
• incomplete_details: None
• instructions: None
• metadata: {}
• model: gpt-4.1-2025-04-14
• object: response
• output: [ResponseOutputMessage(id=‘msg_68a4d42338fc81a1ac7fba8e496af1a1029dfdae1caafc84’, content=[ResponseOutputText(annotations=[], text=‘Hello, world! 👋 How can I help you today?’, type=‘output_text’, logprobs=[])], role=‘assistant’, status=‘completed’, type=‘message’)]
• parallel_tool_calls: True
• temperature: 1.0
• tool_choice: auto
• tools: []
• top_p: 1.0
• background: False
• max_output_tokens: None
• max_tool_calls: None
• previous_response_id: None
• prompt: None
• prompt_cache_key: None
• reasoning: Reasoning(effort=None, generate_summary=None, summary=None)
• safety_identifier: None
• service_tier: default
• status: completed
• text: ResponseTextConfig(format=ResponseFormatText(type=‘text’), verbosity=‘medium’)
• top_logprobs: 0
• truncation: disabled
• usage: ResponseUsage(input_tokens=11, input_tokens_details=InputTokensDetails(cached_tokens=0), output_tokens=14, output_tokens_details=OutputTokensDetails(reasoning_tokens=0), total_tokens=25)
• user: None
• store: True

anthropic

from anthropic import Anthropic

client = Anthropic()

client.messages.create(
    model="claude-3-haiku-20240307",
    max_tokens=1024,
    messages=[ {"role": "user", "content": "Hello, world!"} ]
)

Hello! It’s great to meet you. I’m an AI assistant created by Anthropic. I’m here to help with a wide variety of tasks, from analysis and research to creative projects and casual conversation. Please let me know if there’s anything I can assist you with.

• id: msg_01LVFGsTHhwM65ESmydXgm3o
• content: [{'citations': None, 'text': "Hello! It's great to meet you. I'm an AI assistant created by Anthropic. I'm here to help with a wide variety of tasks, from analysis and research to creative projects and casual conversation. Please let me know if there's anything I can assist you with.", 'type': 'text'}]
• model: claude-3-haiku-20240307
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0}, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 11, 'output_tokens': 60, 'server_tool_use': None, 'service_tier': 'standard'}

As we can see both APIs use the exact same message structure.

mk_msg

Ok, let’s build the first version of mk_msg to handle this case

def mk_msg(content:str, role:str="user")->dict:
    "Create an OpenAI/Anthropic compatible message."
    return dict(role=role, content=content)

Let’s test it out with the OpenAI API. To do that we’ll need to setup two things:

• install the openai SDK by running pip install openai
• add your openai api key to your env vars export OPENAI_API_KEY="YOUR_OPEN_API_KEY"

oa_cli = OpenAI()

r = oa_cli.responses.create(
  model="gpt-4o-mini",
  input=[mk_msg("Hello, world!")]
)
r.output_text

'Hello! How can I assist you today?'

Now, let’s test out mk_msg on the Anthropic API. To do that we’ll need to setup two things:

• install the openai SDK by running pip install anthropic
• add your anthropic api key to your env vars export ANTHROPIC_API_KEY="YOUR_ANTHROPIC_API_KEY"

a_cli = Anthropic()

r = a_cli.messages.create(
    model="claude-3-haiku-20240307",
    max_tokens=1024,
    messages=[mk_msg("Hello, world!")]
)
r.content[0].text

"Hello! I'm an AI assistant created by Anthropic. It's nice to meet you. How can I help you today?"

So far so good!

Helper Functions

Before going any further, let’s create some helper functions to make it a little easier to call the OpenAI and Anthropic APIs. We’re going to be making a bunch of API calls to test our code and typing the full expressions out each time will become a little tedious. These functions won’t be included in the final package.

def openai_chat(msgs: list)->tuple:
    "call the openai chat responses endpoint with `msgs`."
    r = oa_cli.responses.create(model="o4-mini", input=msgs)
    return r, r.output_text

Let’s double check that mk_msg still works with our simple text example from before.

_, text = openai_chat([mk_msg("Hello, world!")])
text

'Hello there! How can I assist you today?'

def anthropic_chat(msgs: list)->tuple:
    "call the anthropic messages endpoint with `msgs`."
    r = a_cli.messages.create(model="claude-sonnet-4-20250514", max_tokens=1024, messages=msgs)
    return r, r.content[0].text

and Anthropic…

_, text = anthropic_chat([mk_msg("Hello, world!")])
text

'Hello! Nice to meet you. How are you doing today? Is there anything I can help you with?'

Images

Ok, let’s see how both APIs handle image messages.

openai

import base64, httpx

img_url = "https://claudette.answer.ai/index_files/figure-html/cell-35-output-1.jpeg"

mtype = "image/jpeg"
img_content = httpx.get(img_url).content

img = base64.b64encode(img_content).decode("utf-8")

client = OpenAI()
r = client.responses.create(
    model="gpt-4o-mini",
    input=[
        {
            "role":"user",
            "content": [
                {"type":"input_text","text":"What's in this image?"},
                {"type":"input_image","image_url":f"data:image/jpeg;base64,{img}"},
            ],
        }
    ],
)
r.output_text

'The image features a puppy lying on the grass near a cluster of purple flowers. The puppy has a brown and white coat, with large, expressive eyes and floppy ears, giving it an adorable appearance.'

anthropic

mtype = "image/jpeg"
img = base64.b64encode(img_content).decode("utf-8")

client = Anthropic()
r = client.messages.create(
    model="claude-3-haiku-20240307",
    max_tokens=1024,
    messages=[
        {
            "role":"user",
            "content": [
                {"type":"text","text":"What's in this image?"},
                {"type":"image","source":{"type":"base64","media_type":mtype,"data":img}}
            ],
        }
    ],
)
r.content[0].text

'The image shows a close-up of a cute puppy lying in the grass. The puppy appears to be a Cavalier King Charles Spaniel, with a fluffy brown and white coat. The puppy is looking directly at the camera with a friendly, curious expression. In the background, there are some purple daisy-like flowers blooming, adding a nice natural setting to the scene.'

Both APIs format images slightly differently and the structure of the message content is a little more complex.

In a text chat, content is a simple string but for a multimodal chat (text+images) we can see that content is a list of dictionaries.

Msg Class

Basics

Let’s create _mk_img to make our code a little DRY’r.

def _mk_img(data:bytes)->tuple:
    "Convert image bytes to a base64 encoded image"
    img = base64.b64encode(data).decode("utf-8")
    mtype = mimetypes.types_map["."+imghdr.what(None, h=data)]
    return img, mtype

To handle the additional complexity of multimodal messages let’s build a Msg class for the content data structure:

{
    "role": "user",
    "content": [{"type": "text", "text": "What's in this image?"}],
}

---

source

Msg

 Msg ()

Helper class to create a message for the OpenAI and Anthropic APIs.

class Msg:
    "Helper class to create a message for the OpenAI and Anthropic APIs."
    pass

As both APIs handle images differently let’s subclass Msg for each API and handle the image formatting in a method called img_msg.

---

source

OpenAiMsg

 OpenAiMsg ()

Helper class to create a message for the OpenAI API.

class OpenAiMsg(Msg):
    "Helper class to create a message for the OpenAI API."
    pass

---

source

AnthropicMsg

 AnthropicMsg ()

Helper class to create a message for the Anthropic API.

class AnthropicMsg(Msg):
    "Helper class to create a message for the Anthropic API."
    pass

Let’s write some helper functions for mk_content to use.

def _is_img(data): return isinstance(data, bytes) and bool(imghdr.what(None, data))

A PDF file should start with %PDF followed by the pdf version %PDF-1.1

def _is_pdf(data): 
    is_byte_pdf = isinstance(data, bytes) and data.startswith(b'%PDF-')
    is_pdf_url = isinstance(data, str) and (
        data.startswith("http") and (data.endswith(".pdf") or 'pdf' in data.split('/'))
    )
    return is_byte_pdf or is_pdf_url

assert _is_pdf("https://arxiv.org/pdf/2301.00001")
assert not _is_pdf("https://arxiv.org/abs/2301.00001")
assert _is_pdf("https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf")
assert not _is_pdf("Hi /pdf/")

We create an appropriate type based on content:

---

source

Msg.mk_content

 Msg.mk_content (content, text_only=False)

@patch
def mk_content(self:Msg, content, text_only=False)->dict:
    if _is_img(content): return self.img_msg(content)
    if _is_pdf(content): return self.pdf_msg(content)
    if isinstance(content, str): return self.text_msg(content, text_only=text_only)
    return content

…then we call the model with this content:

@patch
def __call__(self:Msg, role:str, content:[list, str], text_only:bool=False, **kw)->dict:
    "Create an OpenAI/Anthropic compatible message with `role` and `content`."
    if content is not None and not isinstance(content, list): content = [content]
    content = [self.mk_content(o, text_only=text_only) for o in content] if content else ''
    return dict(role=role, content=content[0] if text_only else content, **kw)

OpenAI implementations:

---

source

OpenAiMsg.text_msg

 OpenAiMsg.text_msg (s:str, text_only=False)

Convert s to a text message

@patch
def img_msg(self:OpenAiMsg, data:bytes)->dict:
    "Convert `data` to an image message"
    img, mtype = _mk_img(data)
    return {"type": "input_image", "image_url": f"data:{mtype};base64,{img}"}

@patch
def text_msg(self:OpenAiMsg, s:str, text_only=False)->dict: 
    "Convert `s` to a text message"
    if not s.strip(): s='.'
    return s if text_only else {"type": "input_text", "text":s}

---

source

OpenAiMsg.img_msg

 OpenAiMsg.img_msg (data:bytes)

Convert data to an image message

Anthropic implementations:

---

source

AnthropicMsg.text_msg

 AnthropicMsg.text_msg (s:str, text_only=False)

Convert s to a text message

@patch
def img_msg(self:AnthropicMsg, data:bytes)->dict:
    "Convert `data` to an image message"
    img, mtype = _mk_img(data)
    r = {"type": "base64", "media_type": mtype, "data":img}
    return {"type": "image", "source": r}

@patch
def text_msg(self:AnthropicMsg, s:str, text_only=False)->dict: 
    "Convert `s` to a text message"
    if not s.strip(): s='.'
    return s if text_only else {"type": "text", "text":s}

---

source

AnthropicMsg.img_msg

 AnthropicMsg.img_msg (data:bytes)

Convert data to an image message

Update mk_msg to use Msg.

---

source

mk_msg

 mk_msg (content:Union[list,str], role:str='user', *args,
         api:str='openai', **kw)

Create an OpenAI/Anthropic compatible message.

mk_msg(["Hello world", "how are you?"], api='openai')

{ 'content': [ {'text': 'Hello world', 'type': 'input_text'},
               {'text': 'how are you?', 'type': 'input_text'}],
  'role': 'user'}

mk_msg(["Hello world", "how are you?"], api='anthropic')

{ 'content': [ {'text': 'Hello world', 'type': 'text'},
               {'text': 'how are you?', 'type': 'text'}],
  'role': 'user'}

msg = mk_msg([img_content, "describe this picture"], api="openai")
_, text = openai_chat([msg])
text

'This is an outdoor scene featuring a young puppy resting in a garden setting. Key details include:\n\n• Breed characteristics: The puppy has the look of a Cavalier King Charles Spaniel  \n• Coloring: Predominantly white coat with rich chestnut-brown patches on the ears, around the eyes, and a spot on the body  \n• Facial features: Large, dark brown eyes; a small black nose; gentle, curious expression  \n• Pose: Lying down on green grass, front legs stretched forward, head lowered slightly as if peeking out  \n• Surroundings:  \n  – A cluster of small, purple daisy-like flowers (possibly asters) to one side  \n  – A wooden structure or planter partially visible behind the puppy, providing shade and a cozy nook  \n• Lighting and mood: Soft, natural daylight with a calm, tranquil atmosphere—conveys a sense of innocence and playfulness  \n\nOverall, the image captures an adorable moment of a floppy-eared puppy relaxing amid a patch of grass and blooms.'

msg = mk_msg([img_content, "describe this picture"], api="anthropic")
_, text = anthropic_chat([msg])
text

'This is an adorable photograph of a young puppy, likely a Cavalier King Charles Spaniel, lying on grass. The puppy has beautiful reddish-brown and white markings - with white on the face, chest, and paws, and rich brown coloring around the ears and eyes. The puppy has sweet, dark eyes and long, silky ears typical of the breed.\n\nThe setting appears to be a garden, with purple flowers (possibly asters or similar blooms) visible in the background, creating a lovely natural backdrop. The puppy is positioned on green grass, and there seems to be a brick or stone structure behind the flowers. The lighting is soft and natural, giving the image a warm, peaceful quality that perfectly captures the innocence and charm of this young dog.'

PDFs

What about chatting with PDFs? Unfortunately, OpenAI’s message completions API doesn’t offer PDF support at the moment, but Claude does.

Under the hood, Claude extracts the text from the PDF and converts each page to an image. This means you can ask Claude about any text, pictures, charts, and tables in the PDF. Here’s an example from the Claude docs. Overall the message structure is pretty similar to an image message.

pdf_url = "https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf"
pdf_data = base64.standard_b64encode(httpx.get(pdf_url).content).decode("utf-8")
client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-3-5-sonnet-20241022", max_tokens=1024,
    messages=[{
        "role": "user",
        "content": [
            {
                "type": "document",
                "source": { "type": "base64", "media_type": "application/pdf", "data": pdf_data }
            },
            {
                "type": "text",
                "text": "Which model has the highest human preference win rates across each use-case?"
            }
        ]
    }]
)

The Anthropic API has since offered an option for PDFs that can be accessed online via url.

client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-opus-4-20250514",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document",
                    "source": {
                        "type": "url",
                        "url": "https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf"
                    }
                },
                {
                    "type": "text",
                    "text": "What are the key findings in this document?"
                }
            ]
        }
    ],
)

Let’s create a method that converts a byte string to the base64 encoded string that Anthropic expects.

def _mk_pdf(data:bytes)->str:
    "Convert pdf bytes to a base64 encoded pdf"
    return base64.standard_b64encode(data).decode("utf-8")

We add a pdf_msg method to AnthropicMsg that uses _mk_pdf.

---

source

AnthropicMsg.pdf_msg

 AnthropicMsg.pdf_msg (data:bytes|str)

Convert data to a pdf message

@patch
def pdf_msg(self:AnthropicMsg, data: bytes | str) -> dict:
    "Convert `data` to a pdf message"
    if isinstance(data, bytes):
        r = {"type": "base64", "media_type": "application/pdf", "data":_mk_pdf(data)}
    elif isinstance(data, str):
        r = {"type": "url", "url": data}
    return {"type": "document", "source": r}

Let’s test our changes on a financial report.

pdf = Path('financial_report.pdf').read_bytes()
msg = mk_msg([pdf, "what was the average monthly revenue for product D?"], api="anthropic")
_, text = anthropic_chat([msg])
text

'Looking at the Product D chart on page 5, I can see the monthly revenue values for each month of 2023. Let me calculate the average:\n\nFrom the chart, the approximate monthly revenues for Product D are:\n- January: $900\n- February: $500\n- March: $400\n- April: $700\n- May: $800\n- June: $900\n- July: $1000\n- August: $1050\n- September: $1200\n- October: $1300\n- November: $1300\n- December: $1300\n\nTotal revenue: $11,350\nNumber of months: 12\n\nAverage monthly revenue for Product D = $11,350 ÷ 12 = **$945.83**'

pdf_url = "https://arxiv.org/pdf/2506.18880"
msg = mk_msg([pdf_url, "What were the three types of generalization the authors of this paper looked at?"], api="anthropic")
_, text = anthropic_chat([msg])
text

'Based on the paper, the authors examined three types of generalization, inspired by Boden\'s typology of creativity:\n\n1. **Exploratory Generalization** - Assessing whether models can apply known problem-solving skills to more complex instances within the same problem domain. For example, counting rectangles in an octagon (training) versus a dodecagon (test). This tests if models can faithfully extend a single reasoning strategy beyond the complexity range seen during training.\n\n2. **Compositional Generalization** - Evaluating the ability to combine distinct reasoning skills, previously learned in isolation, to solve novel problems that require integrating these skills in new and coherent ways. For example, combining GCD computation with polynomial root-finding to solve problems that require both skills working together synergistically.\n\n3. **Transformative Generalization** - Testing whether models can adopt novel, often unconventional strategies by moving beyond familiar approaches to solve problems more effectively. This involves abandoning a familiar but ineffective strategy in favor of a qualitatively different and more efficient approach - essentially requiring a "jump out of the box" or creative reframing. For instance, replacing brute-force enumeration with a subtractive counting method that overcounts and then removes invalid cases.\n\nThe authors designed OMEGA to systematically evaluate these three axes of out-of-distribution generalization using carefully constructed training-test pairs that isolate each specific reasoning capability.'

Conversation

LLMs are stateless. To continue a conversation we need to include the entire message history in every API call. By default the role in each message alternates between user and assistant.

Let’s add a method that alternates the roles for us and then calls mk_msgs.

def mk_msgs(msgs: list, *args, api:str="openai", **kw) -> list:
    "Create a list of messages compatible with OpenAI/Anthropic."
    if isinstance(msgs, str): msgs = [msgs]
    return [mk_msg(o, ('user', 'assistant')[i % 2], *args, api=api, **kw) for i, o in enumerate(msgs)]

mk_msgs(["Hello", "Some assistant response", "tell me a joke"])

[{'role': 'user', 'content': 'Hello'},
 {'role': 'assistant', 'content': 'Some assistant response'},
 {'role': 'user', 'content': 'tell me a joke'}]

SDK Objects

To make our lives even easier, it would be nice if mk_msg could format the SDK objects returned from a previous chat so that we can pass them straight to mk_msgs.

The OpenAI SDK accepts objects like ChatCompletion as messages. Anthropic is different and expects every message to have the role, content format that we’ve seen so far.

---

source

Msg.__call__

 Msg.__call__ (role:str, content:[<class'list'>,<class'str'>],
               text_only:bool=False, **kw)

Create an OpenAI/Anthropic compatible message with role and content.

---

source

AnthropicMsg.find_block

 AnthropicMsg.find_block (r)

Find the message in r.

---

source

AnthropicMsg.is_sdk_obj

 AnthropicMsg.is_sdk_obj (r)

Check if r is an SDK object.

---

source

OpenAiMsg.find_block

 OpenAiMsg.find_block (r)

Find the message in r.

---

source

OpenAiMsg.is_sdk_obj

 OpenAiMsg.is_sdk_obj (r)

Check if r is an SDK object.

---

source

mk_msgs

 mk_msgs (msgs:list, *args, api:str='openai', **kw)

Create a list of messages compatible with OpenAI/Anthropic.

Let’s test our changes.

msgs = ["tell me a joke"]
r, text = openai_chat(mk_msgs(msgs))
text

'Why did the AI go to art school?  \nTo learn how to draw better conclusions!'

msgs += [r, "tell me another joke that's similar to your first joke"]
mm = mk_msgs(msgs)
mm

[{'role': 'user', 'content': 'tell me a joke'},
 ResponseReasoningItem(id='rs_689f894eb37c81a195cd499fd5276b2a08571b67e9050be5', summary=[], type='reasoning', content=None, encrypted_content=None, status=None),
 ResponseOutputMessage(id='msg_689f8951d0b881a1b8f69315efed0efa08571b67e9050be5', content=[ResponseOutputText(annotations=[], text='Why did the AI go to art school?  \nTo learn how to draw better conclusions!', type='output_text', logprobs=[])], role='assistant', status='completed', type='message'),
 {'role': 'user',
  'content': "tell me another joke that's similar to your first joke"}]

r, text = openai_chat(mm)
text

'Why did the AI enroll in a sculpting class?  \nTo improve its model-shaping skills!'

Usage

To make msglm a little easier to use let’s create OpenAI and Anthropic wrappers for mk_msg and mk_msgs.

mk_msg_anthropic = partial(mk_msg, api="anthropic")
mk_msgs_anthropic = partial(mk_msgs, api="anthropic")

If you’re using OpenAI you should be able to use the import below

from msglm import mk_msg_openai as mk_msg, mk_msgs_openai as mk_msgs

Similarily for Anthropic

from msglm import mk_msg_anthropic as mk_msg, mk_msgs_anthropic as mk_msgs

Extra features

Caching

Anthropic currently offers prompt caching, which can reduce cost and latency.

To cache a message, we simply add a cache_control field to our content as shown below.

{
    "role": "user",
    "content": [
        {
            "type": "text",
            "text": "Hello, can you tell me more about the solar system?",
            "cache_control": {"type": "ephemeral"}
        }
    ]
}

Let’s update our mk_msg and mk_msgs Anthropic wrappers to support caching.

---

source

mk_msgs_anthropic

 mk_msgs_anthropic (*args, cache=False, ttl=None,
                    cache_last_ckpt_only=False, api:str='openai')

Create a list of Anthropic compatible messages.

---

source

mk_msg_anthropic

 mk_msg_anthropic (*args, cache=False, ttl=None, role:str='user',
                   api:str='openai')

Create an Anthropic compatible message.

Let’s see caching in action

mk_msg_anthropic("Don't cache my message")

{'content': "Don't cache my message", 'role': 'user'}

mk_msg_anthropic("Please cache my message", cache=True)

{ 'content': [ { 'cache_control': {'type': 'ephemeral'},
                 'text': 'Please cache my message',
                 'type': 'text'}],
  'role': 'user'}

mk_msg_anthropic("Cache for 1 hour", cache=True, ttl="1h")

{ 'content': [ { 'cache_control': {'ttl': '1h', 'type': 'ephemeral'},
                 'text': 'Cache for 1 hour',
                 'type': 'text'}],
  'role': 'user'}

Citations

The Anthropic API provides detailed citations when answering questions about documents.

When citations are enabled a citations block like the one below will be included in the response.

{
  "content": [
    { "type": "text", "text": "According to the document, " },
    {
      "type": "text", "text": "the grass is green",
      "citations": [{
        "type": "char_location",
        "cited_text": "The grass is green.",
        "document_index": 0, "document_title": "Example Document",
        "start_char_index": 0, "end_char_index": 20
      }]
    }
  ]
}

To enable citations you need to create an Anthropic document with the following structure.

{
    "type": "document",
    "source": {...},
    "title": "Document Title", # optional
    "context": "Context about the document that will not be cited from", # optional
    "citations": {"enabled": True}
}

Currently Anthropic supports citations on 3 document types: - text - pdfs - custom

A text document has the following source structure.

{"type": "text", "media_type": "text/plain", "data": "Plain text content..."}

Here’s the source structure for a pdf.

{"type": "base64", "media_type": "application/pdf", "data": b64_enc_data}

Finally, here’s the source structure for a custom document.

{
  "type": "content",
  "content": [
    {"type": "text", "text": "First chunk"},
    {"type": "text", "text": "Second chunk"}
  ]
}

---

source

mk_ant_doc

 mk_ant_doc (content, title=None, context=None, citation=True, **kws)

Create an Anthropic document.

Here’s how you would implement the example from the citation’s docs.

doc = mk_ant_doc("The grass is green. The sky is blue.", title="My Document", context="This is a trustworthy document.")
mk_msg([doc, "What color is the grass and sky?"])

{ 'content': [ { 'citations': {'enabled': True},
                 'context': 'This is a trustworthy document.',
                 'source': { 'data': 'The grass is green. The sky is blue.',
                             'media_type': 'text/plain',
                             'type': 'text'},
                 'title': 'My Document',
                 'type': 'document'},
               { 'text': 'What color is the grass and sky?',
                 'type': 'input_text'}],
  'role': 'user'}