Google Gemini Chat Format (Generate Content)

📝 Introduction

Google Gemini API supports generating content using images, audio, code, tools, etc. Given input GenerateContentRequest generates model responses. Supports text generation, visual understanding, audio processing, long context, code execution, JSON schema, function calling, and many other features.

🤖 Supported Models

Currently supported models include:

💡 Request Examples

Basic Text Chat ✅

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{
        "parts":[{"text": "Write a story about a magic backpack."}]
        }]
       }'

Image Analysis Chat ✅

# Use temporary file to save base64 encoded image data
TEMP_B64=$(mktemp)
trap 'rm -f "$TEMP_B64"' EXIT
base64 $B64FLAGS $IMG_PATH > "$TEMP_B64"

# Use temporary file to save JSON payload
TEMP_JSON=$(mktemp)
trap 'rm -f "$TEMP_JSON"' EXIT

cat > "$TEMP_JSON" << EOF
{
  "contents": [{
    "parts":[
      {"text": "Tell me about this instrument"},
      {
        "inline_data": {
          "mime_type":"image/jpeg",
          "data": "$(cat "$TEMP_B64")"
        }
      }
    ]
  }]
}
EOF

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d "@$TEMP_JSON"

Function Calling ✅

cat > tools.json << EOF
{
  "function_declarations": [
    {
      "name": "enable_lights",
      "description": "Turn on the lighting system."
    },
    {
      "name": "set_light_color",
      "description": "Set the light color. Lights must be enabled for this to work.",
      "parameters": {
        "type": "object",
        "properties": {
          "rgb_hex": {
            "type": "string",
            "description": "The light color as a 6-digit hex string, e.g. ff0000 for red."
          }
        },
        "required": [
          "rgb_hex"
        ]
      }
    },
    {
      "name": "stop_lights",
      "description": "Turn off the lighting system."
    }
  ]
} 
EOF

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
  -H 'Content-Type: application/json' \
  -d @<(echo '
  {
    "system_instruction": {
      "parts": {
        "text": "You are a helpful lighting system bot. You can turn lights on and off, and you can set the color. Do not perform any other tasks."
      }
    },
    "tools": ['$(cat tools.json)'],

    "tool_config": {
      "function_calling_config": {"mode": "auto"}
    },

    "contents": {
      "role": "user",
      "parts": {
        "text": "Turn on the lights please."
      }
    }
  }
')

JSON Schema Response ✅

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
-H 'Content-Type: application/json' \
-d '{
    "contents": [{
      "parts":[
        {"text": "List 5 popular cookie recipes"}
        ]
    }],
    "generationConfig": {
        "response_mime_type": "application/json",
        "response_schema": {
          "type": "ARRAY",
          "items": {
            "type": "OBJECT",
            "properties": {
              "recipe_name": {"type":"STRING"},
            }
          }
        }
    }
}'

Audio Processing 🟡

# Use File API to upload audio data to API request
# Use base64 inline_data to upload audio data to API request
if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then
  B64FLAGS="--input"
else
  B64FLAGS="-w0"
fi
AUDIO_B64=$(base64 $B64FLAGS "$AUDIO_PATH")

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{
    "contents": [{
      "parts": [
        {"text": "Please describe this audio file."},
        {"inline_data": {"mime_type": "audio/mpeg", "data": "'$AUDIO_B64'"}}
      ]
    }]
  }'

Video Processing 🟡

# Use File API to upload video data to API request
# Use base64 inline_data to upload video data to API request
if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then
  B64FLAGS="--input"
else
  B64FLAGS="-w0"
fi
VIDEO_B64=$(base64 $B64FLAGS "$VIDEO_PATH")

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{
    "contents": [{
      "parts": [
        {"text": "Transcribe the audio from this video and provide visual descriptions."},
        {"inline_data": {"mime_type": "video/mp4", "data": "'$VIDEO_B64'"}}
      ]
    }]
  }'

PDF Processing 🟡

MIME_TYPE=$(file -b --mime-type "${PDF_PATH}")
# 使用 base64 inline_data 上传 PDF 文件到 API 请求
if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then
  B64FLAGS="--input"
else
  B64FLAGS="-w0"
fi
PDF_B64=$(base64 $B64FLAGS "$PDF_PATH")

echo $MIME_TYPE

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{
    "contents": [{
      "parts": [
        {"text": "Can you add a few more lines to this poem?"},
        {"inline_data": {"mime_type": "application/pdf", "data": "'$PDF_B64'"}}
      ]
    }]
  }'

Chat Dialog ✅

curl https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [
        {"role":"user",
         "parts":[{
           "text": "Hello"}]},
        {"role": "model",
         "parts":[{
           "text": "Great to meet you. What would you like to know?"}]},
        {"role":"user",
         "parts":[{
           "text": "I have two dogs in my house. How many paws are in my house?"}]},
      ]
    }'

Streaming Response ✅

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:streamGenerateContent?alt=sse&key=$API_KEY" \
    -H 'Content-Type: application/json' \
    --no-buffer \
    -d '{
      "contents": [{
        "parts": [{"text": "写一个关于魔法背包的故事"}]
      }]
    }'

Code Execution ✅

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{
        "parts": [{"text": "计算斐波那契数列的第10项"}]
      }],
      "tools": [{
        "codeExecution": {}
      }]
    }'

Generation Config ✅

curl https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
        "contents": [{
            "parts":[
                {"text": "Explain how AI works"}
            ]
        }],
        "generationConfig": {
            "stopSequences": [
                "Title"
            ],
            "temperature": 1.0,
            "maxOutputTokens": 800,
            "topP": 0.8,
            "topK": 10
        }
    }'

Safety Settings ✅

echo '{
    "safetySettings": [
        {"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_ONLY_HIGH"},
        {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_MEDIUM_AND_ABOVE"}
    ],
    "contents": [{
        "parts":[{
            "text": "'I support Martians Soccer Club and I think Jupiterians Football Club sucks! Write a ironic phrase about them.'"}]}]}' > request.json

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d @request.json

System Instruction ✅

curl "https://computevault.unodetech.xyz/v1beta/models/gemini-3-pro-preview:generateContent?key=$API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "system_instruction": {
    "parts":
      { "text": "You are a cat. Your name is Neko."}},
    "contents": {
      "parts": {
        "text": "Hello there"}}}'

📮 Request

Endpoints

Generate Content

POST https://computevault.unodetech.xyz/v1beta/{model=models/*}:generateContent

Stream Generate Content

POST https://computevault.unodetech.xyz/v1beta/{model=models/*}:streamGenerateContent

Authentication Method

Include API key in the request URL:

?key=$API_KEY

Where $API_KEY is your Google AI API key.

Path Parameters

model

• Type: string
• Required: yes

The name of the model to generate completions for.

Format: models/{model}, e.g. models/gemini-3-pro-preview

Request Body Parameters

contents

• Type: array
• Required: yes

The content of the current conversation with the model. For a single-turn query, this is a single instance. For chat-like multi-turn queries, this is a repeated field containing the conversation history and the latest request.

Content object properties:

Part object properties:

InlineData object properties:

FileData object properties:

tools

• Type: array
• Required: no

A list of tools that the model might use to generate the next response. Supported tools include functions and code execution.

Tool object properties:

FunctionDeclaration object properties:

FunctionCall object properties:

FunctionResponse object properties:

ExecutableCode object properties:

CodeExecutionResult object properties:

CodeExecution object properties:

toolConfig

• Type: object
• Required: no

Tool configuration for any tools specified in the request.

ToolConfig object properties:

FunctionCallingConfig object properties:

FunctionCallingMode enum values:

• MODE_UNSPECIFIED: Default mode, model decides whether to call a function
• AUTO: Model automatically decides when to call a function
• ANY: Model must call a function
• NONE: Model cannot call a function

safetySettings

• Type: array
• Required: no

A list of SafetySetting instances to filter out unsafe content.

SafetySetting object properties:

HarmCategory enum values:

• HARM_CATEGORY_HARASSMENT: Harassment content
• HARM_CATEGORY_HATE_SPEECH: Hate speech and content
• HARM_CATEGORY_SEXUALLY_EXPLICIT: Explicitly sexual content
• HARM_CATEGORY_DANGEROUS_CONTENT: Dangerous content
• HARM_CATEGORY_CIVIC_INTEGRITY: Content that might be used to undermine civic integrity

HarmBlockThreshold enum values:

• BLOCK_LOW_AND_ABOVE: Allows content with a NEGLIGIBLE score to be published
• BLOCK_MEDIUM_AND_ABOVE: Allows content with a NEGLIGIBLE and LOW score to be published
• BLOCK_ONLY_HIGH: Allows content with a NEGLIGIBLE, LOW, and MEDIUM risk level to be published
• BLOCK_NONE: Allows all content
• OFF: Turns off safety filters

Complete HarmBlockThreshold enum values:

• HARM_BLOCK_THRESHOLD_UNSPECIFIED: Threshold not specified
• BLOCK_LOW_AND_ABOVE: Blocks harmful content with a probability of medium or higher, only allowing NEGLIGIBLE level content
• BLOCK_MEDIUM_AND_ABOVE: Blocks harmful content with a probability of medium or higher, allowing NEGLIGIBLE and LOW level content
• BLOCK_ONLY_HIGH: Only blocks harmful content with a high probability, allowing NEGLIGIBLE, LOW, and MEDIUM level content
• BLOCK_NONE: Does not block any content, allowing all levels
• OFF: Completely turns off safety filters

systemInstruction

• Type: object (Content)
• Required: no

System instruction set by the developer. Currently only supports text.

generationConfig

• Type: object
• Required: no

Model generation and output configuration options.

GenerationConfig object properties:

Supported MIME types:

• text/plain: (default) Text output
• application/json: JSON response
• text/x.enum: ENUM as string response

Modality enum values:

• TEXT: Indicates model should return text
• IMAGE: Indicates model should return image
• AUDIO: Indicates model should return audio

Schema object properties:

Type enum values:

• TYPE_UNSPECIFIED: Type not specified
• STRING: String type
• NUMBER: Number type
• INTEGER: Integer type
• BOOLEAN: Boolean type
• ARRAY: Array type
• OBJECT: Object type

Supported programming languages (ExecutableCode):

• LANGUAGE_UNSPECIFIED: Language not specified
• PYTHON: Python programming language

Code execution result enum (Outcome):

• OUTCOME_UNSPECIFIED: Result not specified
• OUTCOME_OK: Code execution successful
• OUTCOME_FAILED: Code execution failed
• OUTCOME_DEADLINE_EXCEEDED: Code execution timed out

cachedContent

• Type: string
• Required: no

The name of cached content, used as context for providing predictions. Format: cachedContents/{cachedContent}

📥 Response

GenerateContentResponse

Answer from models that support multiple candidate answers. The system reports safety ratings and content filtering for the prompt and each candidate.

candidates

• Type: array
• Description: List of candidate answers from the model

Candidate object properties:

FinishReason enum values:

• STOP: Natural stopping point or provided stop sequence for the model
• MAX_TOKENS: Maximum token limit specified in the request reached
• SAFETY: Answer candidate content marked for safety reasons
• RECITATION: Answer candidate content marked for recitation reasons
• LANGUAGE: Answer candidate content marked for using unsupported language
• OTHER: Reason unknown
• BLOCKLIST: Token generation operation stopped because content contains prohibited words
• PROHIBITED_CONTENT: Token generation operation stopped because content might contain prohibited content
• SPII: Token generation operation stopped because content might contain sensitive personal information
• MALFORMED_FUNCTION_CALL: Model-generated function call invalid
• IMAGE_SAFETY: Token generation stopped because generated image violated safety rules

promptFeedback

• Type: object
• Description: Prompt feedback related to content filtering

PromptFeedback object properties:

BlockReason enum values:

• BLOCK_REASON_UNSPECIFIED: Default value, this value is not used
• SAFETY: System blocked prompt due to safety reasons
• OTHER: Prompt blocked due to unknown reasons
• BLOCKLIST: System blocked this prompt because it contained terms in the blocklist
• PROHIBITED_CONTENT: System blocked this prompt because it contained prohibited content
• IMAGE_SAFETY: Candidate image blocked because it generated unsafe content

usageMetadata

• Type: object
• Description: Metadata about token usage for the generation request

UsageMetadata object properties:

modelVersion

• Type: string
• Description: Model version used to generate the answer

responseId

• Type: string
• Description: ID for each response

Full response example

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "text": "你好！我是 Gemini，一个由 Google 开发的人工智能助手。我可以帮助您解答问题、提供信息、协助写作、代码编程等多种任务。请告诉我有什么可以为您效劳的！"
          }
        ],
        "role": "model"
      },
      "finishReason": "STOP",
      "index": 0,
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE",
          "blocked": false
        },
        {
          "category": "HARM_CATEGORY_HATE_SPEECH", 
          "probability": "NEGLIGIBLE",
          "blocked": false
        },
        {
          "category": "HARM_CATEGORY_HARASSMENT",
          "probability": "NEGLIGIBLE",
          "blocked": false
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
          "probability": "NEGLIGIBLE",
          "blocked": false
        }
      ],
      "tokenCount": 47
    }
  ],
  "promptFeedback": {
    "safetyRatings": [
      {
        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
        "probability": "NEGLIGIBLE"
      },
      {
        "category": "HARM_CATEGORY_HATE_SPEECH",
        "probability": "NEGLIGIBLE"
      }
    ]
  },
  "usageMetadata": {
    "promptTokenCount": 4,
    "candidatesTokenCount": 47,
    "totalTokenCount": 51,
    "promptTokensDetails": [
      {
        "modality": "TEXT",
        "tokenCount": 4
      }
    ],
    "candidatesTokensDetails": [
      {
        "modality": "TEXT", 
        "tokenCount": 47
      }
    ]
  },
  "modelVersion": "gemini-3-pro-preview",
  "responseId": "response-12345"
}

🔧 Advanced Features

Safety Ratings

SafetyRating object properties:

HarmProbability enum values:

• NEGLIGIBLE: Harmful probability negligible
• LOW: Harmful probability low
• MEDIUM: Harmful probability medium
• HIGH: Harmful probability high

Citation Metadata

CitationMetadata object properties:

CitationSource object properties:

Code Execution

When code execution tools are enabled, the model can generate and execute code to solve problems.

Code execution example response:

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "text": "我来计算斐波那契数列的第10项："
          },
          {
            "executableCode": {
              "language": "PYTHON",
              "code": "def fibonacci(n):\n    if n <= 1:\n        return n\n    else:\n        return fibonacci(n-1) + fibonacci(n-2)\n\nresult = fibonacci(10)\nprint(f'第10项斐波那契数是: {result}')"
            }
          },
          {
            "codeExecutionResult": {
              "outcome": "OK",
              "output": "第10项斐波那契数是: 55"
            }
          },
          {
            "text": "所以斐波那契数列的第10项是55。"
          }
        ],
        "role": "model"
      },
      "finishReason": "STOP"
    }
  ]
}

Grounding

GroundingMetadata object properties:

GroundingAttribution object properties:

AttributionSourceId object properties:

GroundingPassageId object properties:

SemanticRetrieverChunk object properties:

SearchEntryPoint object properties:

Segment object properties:

RetrievalMetadata object properties:

GroundingChunk object properties:

Web object properties:

GroundingSupport object properties:

Multimodal Processing

Gemini API supports processing multiple modalities of input and output:

Supported input modalities:

• TEXT: Pure text
• IMAGE: Images (JPEG, PNG, WebP, HEIC, HEIF)
• AUDIO: Audio (WAV, MP3, AIFF, AAC, OGG, FLAC)
• VIDEO: Videos (MP4, MPEG, MOV, AVI, FLV, MPG, WEBM, WMV, 3GPP)
• DOCUMENT: Documents (PDF)

ModalityTokenCount object properties:

MediaResolution enum values:

• MEDIA_RESOLUTION_LOW: Low resolution (64 tokens)
• MEDIA_RESOLUTION_MEDIUM: Medium resolution (256 tokens)
• MEDIA_RESOLUTION_HIGH: High resolution (256 tokens for scaling and re-framing)

Thinking Function

ThinkingConfig object properties:

Speech Generation

SpeechConfig object properties:

VoiceConfig object properties:

PrebuiltVoiceConfig object properties:

MultiSpeakerVoiceConfig object properties:

SpeakerVoiceConfig object properties:

Supported language codes:

• zh-CN: Chinese (Simplified)
• en-US: English (US)
• ja-JP: Japanese
• ko-KR: Korean
• fr-FR: French
• de-DE: German
• es-ES: Spanish
• pt-BR: Portuguese (Brazil)
• hi-IN: Hindi
• ar-XA: Arabic
• it-IT: Italian
• tr-TR: Turkish
• vi-VN: Vietnamese
• th-TH: Thai
• ru-RU: Russian
• pl-PL: Polish
• nl-NL: Dutch

Logprobs Results

LogprobsResult object properties:

TopCandidates object properties:

Candidate (Logprobs) object properties:

URL Retrieval Function

UrlRetrievalMetadata object properties:

UrlRetrievalContext object properties:

UrlContextMetadata object properties:

UrlMetadata object properties:

UrlRetrievalStatus enum values:

• URL_RETRIEVAL_STATUS_SUCCESS: URL retrieval successful
• URL_RETRIEVAL_STATUS_ERROR: URL retrieval failed due to an error

Complete Harm Categories

HarmCategory enum values:

• HARM_CATEGORY_UNSPECIFIED: Category not specified
• HARM_CATEGORY_DEROGATORY: PaLM - Negative or harmful comments targeting identity and/or protected attributes
• HARM_CATEGORY_TOXICITY: PaLM - Rude, impolite, or profane content
• HARM_CATEGORY_VIOLENCE: PaLM - Scenarios depicting violence against individuals or groups
• HARM_CATEGORY_SEXUAL: PaLM - References to sexual behavior or other explicit content
• HARM_CATEGORY_MEDICAL: PaLM - Promoting unverified medical advice
• HARM_CATEGORY_DANGEROUS: PaLM - Dangerous content promotes, encourages, or facilitates harmful behavior
• HARM_CATEGORY_HARASSMENT: Gemini - Harassment content
• HARM_CATEGORY_HATE_SPEECH: Gemini - Hate speech and content
• HARM_CATEGORY_SEXUALLY_EXPLICIT: Gemini - Explicitly sexual content
• HARM_CATEGORY_DANGEROUS_CONTENT: Gemini - Dangerous content
• HARM_CATEGORY_CIVIC_INTEGRITY: Gemini - Content that might be used to undermine civic integrity

HarmProbability enum values:

• HARM_PROBABILITY_UNSPECIFIED: Probability not specified
• NEGLIGIBLE: Harmful probability negligible
• LOW: Harmful probability low
• MEDIUM: Harmful probability medium
• HIGH: Harmful probability high

Modality enum values:

• MODALITY_UNSPECIFIED: Modality not specified
• TEXT: Pure text
• IMAGE: Image
• VIDEO: Video
• AUDIO: Audio
• DOCUMENT: Document, e.g., PDF

MediaResolution enum values:

• MEDIA_RESOLUTION_UNSPECIFIED: Media resolution not set
• MEDIA_RESOLUTION_LOW: Media resolution set to low (64 tokens)
• MEDIA_RESOLUTION_MEDIUM: Media resolution set to medium (256 tokens)
• MEDIA_RESOLUTION_HIGH: Media resolution set to high (using 256 tokens for scaling and re-framing)

UrlRetrievalStatus enum values:

• URL_RETRIEVAL_STATUS_UNSPECIFIED: Default value, this value is not used
• URL_RETRIEVAL_STATUS_SUCCESS: URL retrieval successful
• URL_RETRIEVAL_STATUS_ERROR: URL retrieval failed due to an error

🔍 Error Handling

Common Error Codes

Detailed Error Code Explanations

Error Response Example

{
  "error": {
    "code": 400,
    "message": "Invalid argument: contents",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.BadRequest",
        "fieldViolations": [
          {
            "field": "contents",
            "description": "contents is required"
          }
        ]
      }
    ]
  }
}