Reprocessing Invalid Outputs

[PureSpectrum]

I am working on integrating Google ADK into Mongo databases to serve as a real time analytics agent.

Most of the time, the application works fine, but in some cases, the agent attempts to run Python code directly in the context like "Print(default_api." and this results in a Malformed Function Call and the processing stops. Despite attempts to suppress this via prompting, I am unable to get it to stop. When I send a follow on user message like "Can you please try again?" the agent handles the issue and proceeds.

Separately - after a long running mongo aggregation, the model just returns an empty response. Same as before, I am able to send a follow up message and the chat corrects itself.

In both cases I am able to detect the issue using the "after_model_callback", but I am unable to figure out how I can reject the output/ask it to reprocess programatically.

Schreenshot 1:

Screenshot 2:

Any thoughts on how I can address either issue?

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: You can use the after_model_callback to detect and replace the invalid output with a custom message. While you can't trigger a model re-run directly from the callback in the same turn, your client application can detect this custom message and programmatically send a follow-up prompt (like 'Please try again') to initiate the retry.

---

Hello! Thanks for reaching out with this interesting question. It's a great use case for handling model inconsistencies programmatically.

You are on the right track with using the after_model_callback. This callback is the perfect place to inspect the model's output before it's processed further. Its primary purpose is to allow you to check, log, or even modify the LlmResponse. [1]

Replacing Invalid Output

As you've noted, you can detect both the malformed function call and the empty response in after_model_callback. The key is that if this callback returns a new LlmResponse object, that object will completely replace the original response from the model. If it returns None, the original response is used.

Here is a conceptual example of how you could implement this:

from adk.llm import LlmResponse

def my_after_model_callback(llm_response: LlmResponse) -> LlmResponse | None:
    response_content = llm_response.content
    
    # Check for empty or malformed response
    is_empty = not response_content
    is_malformed = "Print(default_api." in response_content # Example check
    
    if is_empty or is_malformed:
        # Replace the bad response with a specific, structured message
        # that your application can understand.
        return LlmResponse(
            content={
                "error": "model_output_error",
                "message": "The model returned an invalid response. Triggering a retry."
            }
        )
    
    # If the response is valid, do nothing and let ADK proceed.
    return None

# Then, register your callback when running the agent:
# --- Register it during Agent creation ---
my_agent = LlmAgent(
    name="MyCallbackAgent",
    model="gemini-2.0-flash", # Or your desired model
    instruction="Be helpful.",
    # Other agent parameters...
   after_model_callback=my_after_model_callback # Pass the function here
)

Triggering the Retry

The after_model_callback cannot force the model to re-run within the same turn. The agent's execution flow for that turn completes after the callback.

The most effective solution is to handle the retry logic in your client application—the code that calls runner.run().

1. Detect: In your after_model_callback, return a specific, structured response when you detect an error, like in the example above.
2. Inspect: Your client application receives the final output from the agent. It should inspect this output to see if it matches the error structure you defined.
3. Retry: If it detects the error message, your client can then programmatically send a new message to the agent, such as "Can you please try again?", which you've observed already corrects the issue.

Other Considerations

• Error-Specific Callbacks: For the "Malformed Function Call" issue, if it raises a specific exception, you might be able to handle it more directly using the on_model_error_callback available through the ADK Plugins feature. This callback is designed to catch exceptions during the model call and return a fallback response. [2]
• Advanced Retries: If you find this pattern occurs frequently or requires more complex logic, you could explore using a multi-agent setup with a LoopAgent. [3] This would allow you to create a "reviewer" agent that validates the output of a "generator" agent and can loop back to retry generation until the output is valid. This is a more advanced pattern but offers a great deal of control.

I hope this helps you build a more robust solution!

[1] https://google.github.io/adk-docs/callbacks/
[2] https://google.github.io/adk-docs/plugins/
[3] https://google.github.io/adk-docs/agents/multi-agents/

[PureSpectrum]

thanks this is tremendously helpful - for right now I am in the prototyping stage and using the "adk web" runtime. Any idea how I would incorporate

runner.run(agent, callbacks={"after_model": my_after_model_callback})

when running this way?

[hangfei]

--- Register it during Agent creation ---

my_agent = LlmAgent(
name="MyCallbackAgent",
model="gemini-2.0-flash", # Or your desired model
instruction="Be helpful.",
# Other agent parameters...
before_model_callback=my_before_model_logic # Pass the function here
)

or use plugin:

Check out this sample: https://github.com/google/adk-python/tree/main/contributing/samples/hello_world_app