Kisan AI

The problem

Rural Pakistani farmers coordinate machinery rentals and weather decisions through phone calls and WhatsApp voice notes. Most cannot type comfortably in Urdu or Punjabi script, and many struggle to read at all. Existing digital tools assume English literacy and typed input. Voice was not a feature here. It was the only viable way in.

What it does

A farmer taps the microphone and speaks naturally, for example "Mujhe kal subah harvester chahiye, 5 acre gandum kaatni hai." The system transcribes and interprets intent in the user's language, checks the weather for the requested date and location, searches available machinery providers, ranks options and explains the recommendation, creates a confirmed booking only after explicit user confirmation, and responds with spoken audio plus a visual confirmation card.

Built solo during the Google Antigravity Hackathon (Challenge 2: AI Service Orchestrator for the Informal Economy), backend to frontend.

Architecture

• Mobile: React Native, Expo SDK 54, TypeScript, Zustand, TanStack Query, i18next, NativeWind. PCM audio at 16 kHz in, 24 kHz out.
• Backend: FastAPI, Python 3.11+, SQLAlchemy. JWT auth. SQLite in development, PostgreSQL-compatible schema.
• Transport: REST over HTTPS, plus a WebSocket for the voice session.

Two agent runtimes share one tool layer:

• Voice agent (primary): Gemini Live native audio with function calling. The backend acts as a secure proxy between the mobile client and the Gemini Live API. API keys stay server-side, business logic executes locally, every tool invocation is logged.
• Text agent (fallback): Gemini text model running a multi-turn tool-calling loop, up to 8 iterations, with persisted session history.

Nine tools: search_machinery, search_machinery_and_status, get_machinery_details, get_weather, validate_booking_request, create_booking, create_booking_with_confirmation, list_my_bookings, cancel_booking. Machinery search ranks by rating then price. Weather comes from Open-Meteo.

Design decisions

Booking creation is gated behind an explicit confirmation flag. The model cannot commit a booking on its own, no matter how confident it sounds. An agent that books the wrong harvester for a farmer's only harvest window is not a bug you get to apologise for later.

Every tool call is written to an agent_traces table with event type, tool name, input arguments, output result, and timestamp. The mobile app surfaces this as an Agent Trace screen the user can actually read. Full conversation history is persisted for session replay.

Five locales: Urdu in Nastaliq, Pakistani Punjabi in Shahmukhi, Roman Urdu, Roman Punjabi, and English. A script guard sanitizes model output to block Devanagari and Gurmukhi in Pakistani language modes, because the model will happily produce the wrong script for the right language.

Google Antigravity was the IDE used to build the project, not part of the runtime. The live system calls Gemini from the FastAPI backend.

Honest limitations

• Machinery search uses type, price, text, and rating. It is not distance-based PostGIS matching.
• SQLite in development. The schema is PostgreSQL-compatible but not deployed on it.
• OTP verification is simulated. No SMS gateway.
• No payment integration. No JazzCash, Easypaisa, or bank rails.
• No integrated map view or geocoding.
• Booking reminder notifications are not implemented.
• Android-focused prototype.