View : 114

11/08/2026 04:41am

Golang code snippet for handling AI API errors and parsing malformed JSON

Golang The Series EP.168: AI Error Handling — Dealing with Malformed JSON and Unexpected LLM Responses

#Golang

#Go Backend

#AI Error Handling

#LLM

#Retry Loop

#LLM Integration

#Malformed JSON

#Safe Parsing

Welcome to EP.168! In the previous episode (EP.167: Monitoring AI Performance), we set up a monitoring system using Prometheus to measure our system's latency. Today, we're going to tackle one of the biggest pain points that every AI Backend developer has likely encountered: malformed JSON or AI models returning conversational text instead of the structured data we requested.

When writing a standard Web API, if something goes wrong, the server usually spits out an HTTP Status Code immediately, such as 400 Bad Request or 500 Internal Server Error. However, the world of Large Language Models (LLMs) isn't that straightforward. We often face headache-inducing cases like:

  1. HTTP Status 200 OK, but the payload is a Refusal: The model replies with, "I'm sorry, but I cannot answer this question due to safety policies..."

  2. Invalid Structured Output (Malformed JSON): We prompt the model to return JSON so we can use json.Unmarshal into a Go Struct, but the model throws in a Markdown Code Block (json ... ) or forgets to close a brace }, causing the system to crash.

  3. Hallucinations & Out-of-Bounds Values: The model returns correctly formatted data, but the values violate system constraints (e.g., a Score should be between 0.0 - 1.0, but the model sends 99.0).

If we let this unexpected data slip through into our system, it will cause a Panic or immediately break the Business Logic on the Go side. In this episode, we'll look at techniques for catching and handling these errors in Go.

3-Layer Structure for AI Error Handling

To ensure safety, we will implement a 3-layer defense structure:

Plaintext

[User Input]
       │
       ▼
┌─────────────────────────────────┐
│ 1. Pre-validation & Guardrails  │ ───> Catch Prompt Injection / Out-of-scope questions before sending
└────────────────┬────────────────┘
                 │
                 ▼
          [Call LLM API]
                 │
                 ▼
┌─────────────────────────────────┐
│ 2. Structural Parsing & Retry   │ ───> Clean up strings & Catch Malformed JSON
└────────────────┬────────────────┘
                 │
                 ▼
┌─────────────────────────────────┐
│ 3. Semantic Validation          │ ───> Verify Business Rules and numerical bounds
└────────────────┬────────────────┘
                 │
                 ▼
         [Safe Business Logic]

Structure and Go Code for Error Handling with Retry Loop

To deal with models spitting out broken JSON formats, we'll build functions to clean up the text, collect errors, and automatically trigger a Retry Loop if parsing fails.

Part 1: Data Structures, Clean Up Function, and Semantic Validator

Go

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"strings"
	"time"
)

// UserAnalysisResult is the target struct we want to receive from the AI
type UserAnalysisResult struct {
	Sentiment string   `json:"sentiment"`
	Keywords  []string `json:"keywords"`
	Score     float64  `json:"score"`
}

// cleanJSONResponse removes the Markdown Formatting that LLMs often include
func cleanJSONResponse(raw string) string {
	cleaned := strings.TrimSpace(raw)
	// Remove Markdown Code Block
	if strings.HasPrefix(cleaned, "```") {
		lines := strings.Split(cleaned, "\n")
		if len(lines) >= 2 {
			// Remove the first line (```json) and the last line (```)
			cleaned = strings.Join(lines[1:len(lines)-1], "\n")
		}
	}
	return strings.TrimSpace(cleaned)
}

// validateSemantic checks additional business rules after a successful Unmarshal
func (r *UserAnalysisResult) validateSemantic() error {
	// 1. Check if Sentiment is within the acceptable range
	validSentiments := map[string]bool{"positive": true, "negative": true, "neutral": true}
	if !validSentiments[strings.ToLower(r.Sentiment)] {
		return fmt.Errorf("invalid sentiment value: '%s'", r.Sentiment)
	}

	// 2. Check if Score is within the 0.0 - 1.0 range
	if r.Score < 0.0 || r.Score > 1.0 {
		return fmt.Errorf("score out of bounds (0.0 - 1.0): %f", r.Score)
	}

	return nil
}

// simulateLLMCall simulates an AI call, returning potentially problematic formats in early attempts
func simulateLLMCall(attempt int) string {
	switch attempt {
	case 1:
		// Attempt 1: Includes Markdown code block and malformed JSON (fails at Unmarshal)
		return "```json\n{\"sentiment\": \"positive\", \"keywords\": [\"go\", \"ai\"], \"score\": 0.9"
	case 2:
		// Attempt 2: Correct JSON, but invalid Semantic (Score out of bounds)
		return "{\"sentiment\": \"super_happy\", \"keywords\": [\"go\"], \"score\": 99.0}"
	default:
		// Attempt 3: Completely correct response
		return "{\"sentiment\": \"positive\", \"keywords\": [\"golang\", \"ai\"], \"score\": 0.95}"
	}
}

Part 2: Main Safety Control and Execution Runner (main.go)

Go

// SafeAIParseRunner is a function to call LLM + Parse + Validate with a Retry system
func SafeAIParseRunner(ctx context.Context, maxRetries int) (*UserAnalysisResult, error) {
	var lastErr error

	for attempt := 1; attempt <= maxRetries; attempt++ {
		fmt.Printf("🔄 [Attempt %d/%d] Processing AI request...\n", attempt, maxRetries)

		// 1. Call the LLM
		rawOutput := simulateLLMCall(attempt)

		// 2. Clean Up Strings
		cleanedOutput := cleanJSONResponse(rawOutput)

		// 3. Structural Parsing Check
		var result UserAnalysisResult
		if err := json.Unmarshal([]byte(cleanedOutput), &result); err != nil {
			lastErr = fmt.Errorf("JSON Structural Error: %w", err)
			fmt.Printf("❌ Failed: %v\n\n", lastErr)
			time.Sleep(200 * time.Millisecond)
			continue // Skip to the next Retry attempt
		}

		// 4. Semantic Validation Check
		if err := result.validateSemantic(); err != nil {
			lastErr = fmt.Errorf("Semantic Validation Error: %w", err)
			fmt.Printf("❌ Failed: %v\n\n", lastErr)
			time.Sleep(200 * time.Millisecond)
			continue // Skip to the next Retry attempt
		}

		// If all checks pass
		fmt.Printf("✅ Success on attempt %d!\n", attempt)
		return &result, nil
	}

	return nil, fmt.Errorf("Exceeded maximum retries (%d times). Last Error: %w", maxRetries, lastErr)
}

func main() {
	ctx := context.Background()
	result, err := SafeAIParseRunner(ctx, 3)

	if err != nil {
		fmt.Printf("🔴 System Error: %v\n", err)
		// At this point, we can implement Fallback Behavior, e.g., returning a Default Struct
		return
	}

	fmt.Println("\n--- Safe output ready for further processing ---")
	fmt.Printf("Sentiment : %s\n", result.Sentiment)
	fmt.Printf("Keywords  : %v\n", result.Keywords)
	fmt.Printf("Score     : %.2f\n", result.Score)
}

Advanced Techniques: Structured Outputs & Self-Correction Loop

Besides implementing Clean Up and Retry loops on the Go side, we should leverage API features to minimize errors at the source:

  1. JSON Schema Enforcement (Structured Outputs): Define a ResponseFormat with a JSON Schema for the model (e.g., OpenAI Structured Outputs or Gemini Response Schema). This forces the model to reply exactly according to the structure, eliminating malformed JSON issues almost 100%.

  2. Self-Correction Retry (Feedback Loop): When an error occurs on the first try, instead of sending the exact same prompt blindly, append the Go Error Message (e.g., invalid sentiment value: super_happy) to the end of the new prompt to tell the AI: "You made a mistake here, please fix it." This significantly increases the success rate of the next attempt.

🎯 Daily Mission

Try running the code above on your machine and observe the execution and error handling during each attempt in the console.

Challenge: If you encounter a case where the AI replies with a Refusal Text like "I cannot process questions regarding this policy," which will definitely crash json.Unmarshal, how would you write a Custom Error Handler in Go to distinguish between "Standard Malformed JSON" and an "AI Refusal" so the system can handle it appropriately? Give it a try!

🙋‍♂️ FAQ (Frequently Asked Questions)

Why not use Regex to extract just the JSON instead of checking for Markdown Code Blocks?

Using Regex to extract JSON is quite risky and consumes more resources. If the model returns deeply nested JSON or special characters, Regex might fail to match properly. Using simple string manipulation functions to strip out Markdown is much faster and safer in Go.

What should the maxRetries be set to?

Generally, 2 to 3 times is sufficient. Calling an LLM takes time (High Latency). If you retry too many times, the request might block the system and cause a timeout on the client side. It's recommended that once the retry quota is reached, you implement a Fallback system by returning default values to the user.

Do we need to write Semantic Validation for every single field?

Not necessarily. Focus on checking fields that heavily impact the core Business Logic, such as Status, Score, or fields that require further mathematical processing. General text fields (like descriptive notes) can be passed through to save processing time.

📝 Conclusion

When working with AI, we can never trust the output 100%. The techniques we learned today include Pre-validation, building a Clean Up function to strip Markdown from JSON, Semantic Validation to filter out garbage data, and using Retry Loops for automated immediate recovery. Setting up this 3-layer structure will make our Go Backend resilient against LLM unpredictability, allowing us to deploy to Production with confidence.

Coming up next (EP.169): We've been intensely building AI Microservices in Go throughout this series, but many might be wondering, "Why use Go for AI when Python is the industry favorite?" In the next episode, we'll do a head-to-head comparison in "Benchmarking Go vs. Python for AI Pipelines — Analyzing Latency, Concurrency, and Memory Usage." Don't miss it, Gophers!

Follow Superdev Academy on all platforms: