BhashaSetu - The Ethereal Conduit 🌉

BhashaSetu (Sanskrit for "Language Bridge") is an AI-powered document-level machine translation workspace designed specifically for low-resource Indic languages. Built on IndicTrans2, it provides seamless English-to-Indic translation for Malayalam, Telugu, Kannada, Tamil, and Hindi.

• Watch the Streamlit Demo Video Here: Google Drive link
• Watch the Flask + JS Demo Video Here: Google Drive link

✨ Features

• 🌐 Multi-Language Support: English → Malayalam, Telugu, Kannada, Tamil, Hindi
• 🚀 IndicTrans2 Powered: State-of-the-art specialized Indic translation model
• 📊 Real-time Metrics: Translation time tracking and performance monitoring
• 🎨 Elegant UI: Editorial design with glassmorphism and smooth animations
• 📋 One-Click Copy: Copy translations to clipboard
• 🔄 Two Deployment Options: Flask + Vanilla JS or Streamlit

📊 Performance Benchmarks

Language	BLEU	ChrF	TER	METEOR
Kannada	17.82	54.91	76.20	0.3999
Malayalam	15.73	52.87	84.46	0.3530
Telugu	12.54	48.33	84.50	0.3160
Tamil	11.47	50.51	92.45	0.3082

Evaluated on Pralekha dataset test split (1000 samples per language)

🏗️ Architecture

┌─────────────────────────────────────────────────────────────┐
│ BhashaSetu Application │
├─────────────────────────────────────────────────────────────┤
│ ┌─────────────────┐ ┌─────────────────────┐ │
│ │ Frontend UI │◄────────────►│ Backend API │ │
│ │ (HTML/CSS/JS) │ REST API │ (Flask/Streamlit) │ │
│ └─────────────────┘ └──────────┬──────────┘ │
│ │         │
│ ┌─────────▼─────────┐ │
│ │ IndicTrans2-1B │ │
│ │ (Hugging Face) │ │
│ └───────────────────┘ │
│ │         │
│ ┌─────────▼─────────┐ │
│ │ Unicode Indic │ │
│ │ Transliterator │ │
│ └───────────────────┘ │
└─────────────────────────────────────────────────────────────┘

📋 Prerequisites

• Python 3.10+
• Hugging Face Account (for model access)
• 8GB+ RAM (16GB recommended)
• GPU Optional (CUDA-compatible for faster inference)

🔧 Installation

1. Clone the Repository

git clone https://github.com/yourusername/bhashasetu.git
cd bhashasetu

2. Set Up Environment

# Create virtual environment
python -m venv venv```

# Activate virtual environment
# On Windows:
venv\Scripts\activate
# On macOS/Linux:
source venv/bin/activate

3. Install Dependencies

For Flask Version:

pip install -r requirements.txt

For Streamlit Version:

pip install -r requirements-streamlit.txt

4. Configure Hugging Face Token

# Copy the example environment file
cp .env.example .env

# Edit .env and add your Hugging Face token
# Get your token from: https://huggingface.co/settings/tokens
Edit .env:

env
HF_TOKEN=hf_your_actual_token_here
USE_CUDA=0  # Set to 1 if you have GPU
DEBUG=false

5. Model Access Request

• IndicTrans2 requires access approval:
• Visit ai4bharat/indictrans2-en-indic-1B
• Click "Agree and access repository"
• Wait for approval (usually instant)

🚀 Running the Application

Option 1: Streamlit Version (Recommended for Quick Start)

Using the run script

./run_streamlit.sh

# Or directly
streamlit run streamlit/app_streamlit.py
Access at: http://localhost:8501

Features:

• ✅ Single-command startup
• ✅ Built-in UI components
• ✅ Automatic state management
• ✅ Faster development iteration

Option 2: Flask + Frontend Version (Production-Ready)

# Terminal 1: Start Flask Backend
cd backend
python app_flask.py

# Terminal 2: Serve Frontend
cd frontend
python -m http.server 8000
Access at: http://localhost:8000

Features:

• ✅ RESTful API architecture
• ✅ Separation of concerns
• ✅ Scalable for production
• ✅ Custom frontend control

Option 3: Docker Deployment

# Streamlit version
docker-compose -f docker/docker-compose-streamlit.yml up

# Flask version
docker-compose -f docker/docker-compose-flask.yml up

📖 Usage Guide

Basic Guide

• Enter Source Text: Type or paste English text in the left panel
• Select Target Language: Choose from Malayalam, Telugu, Kannada, Tamil, or Hindi
• Choose Style: Select "Formal" or "Casual" translation mode
• Click Translate: Press the "Translate Now" button
• View Results: Translation appears in the right panel with time metrics

🔬 Model Methodology

Model Architecture

BhashaSetu uses IndicTrans2-1B, an encoder-decoder transformer specifically trained on 1.17M English-Indic parallel document pairs from the Pralekha dataset.

Key Specifications:

• Parameters: 1 Billion
• Architecture: Transformer Encoder-Decoder
• Training Data: PIB news bulletins + Mann Ki Baat transcripts
• Context Window: 512 tokens

Translation Pipeline

English Input → Language Tagging → Beam Search Decoding → Script Transliteration → Indic Output
Language Tagging: Input prefixed with eng_Latn {target}_ {script}

Beam Search: 5-beam decoding with length penalty α=1.0

Script Normalization: Hindi-to-Target script conversion via UnicodeIndicTransliterator

Output: Native script rendering

Why Not LLMs?

• We initially tested general-purpose LLMs:
• Qwen2.5-7B: BLEU 0.01, frequent English fallback
• DeepSeek-7B: BLEU 0.0004, script corruption

Failure Reasons:

• Low-resource language limitations in pretraining
• Morphological complexity of Indic scripts
• Long-context instability in decoder-only models
• IndicTrans2's specialized training on parallel Indic corpora provides 1500x improvement over LLM baselines.

📁 Project Structure

bhashasetu/
├── README.md                   # Project documentation
├── .gitignore                  # Git ignore rules
├── .env.example                # Environment variables template
├── requirements.txt            # Flask version dependencies
├── requirements-streamlit.txt  # Streamlit version dependencies
├── setup.sh                    # One-click setup script
├── run_flask.sh               # Flask launcher
├── run_streamlit.sh           # Streamlit launcher
│
├── backend/                    # Flask Backend
│   ├── app_flask.py           # Flask API server
│   ├── model.py               # IndicTrans2 wrapper
│   ├── config.py              # Configuration
│   └── requirements.txt       # Backend dependencies
│
├── frontend/                   # Vanilla JS Frontend
│   ├── index.html             # Main HTML
│   ├── css/
│   │   └── styles.css         # Custom styles
│   └── js/
│       ├── config.js          # Configuration
│       ├── ui.js              # UI manager
│       ├── translation.js     # Translation service
│       └── app.js             # Main application
│
├── streamlit/                  # Streamlit Version
│   └── app_streamlit.py       # Single-file Streamlit app
│
└── docker/                     # Docker Configuration
    ├── Dockerfile.backend
    ├── Dockerfile.streamlit
    ├── docker-compose-flask.yml
    └── docker-compose-streamlit.yml

🔧 Configuration Options

Environment Variables
Variable	Description	             Default
HF_TOKEN	Hugging Face API token	 Required
USE_CUDA	Enable GPU acceleration	 0
DEBUG	    Enable debug mode        false
PORT	    Server port	             5000
MAX_LENGTH	Max translation tokens	 256
BATCH_SIZE	Batch processing size	 4

Language Configuration

Add new languages in config.py or app_streamlit.py:

LANGUAGE_CONFIGS = {
    "NewLanguage": {
        "src": "eng_Latn",
        "tgt": "xxx_Xxxx",
        "translit_src": "hi",
        "translit_tgt": "xx"
    }
}

🐛 Troubleshooting

Common Issues

Issue	                    Solution
Model won't load	        Verify HF_TOKEN in .env file
"Gated model" error	        Accept terms at model page
CUDA out of memory	        Set USE_CUDA=0 or reduce BATCH_SIZE
Material Icons not showing	Check internet connection (fonts loaded from Google)
Slow first translation	    Model loading takes 30-60s; subsequent translations fast
Script showing Devanagari	Transliterator issue; check translit_src/translit_tgt

Debug Mode Enable debug mode for detailed logs:

export DEBUG=true
python backend/app_flask.py

📊 Performance Optimization

GPU Acceleration

# Enable CUDA (requires NVIDIA GPU + CUDA toolkit)
export USE_CUDA=1

Batch Processing

Adjust batch size based on available memory:

# In config.py
BATCH_SIZE = 8  # Increase for better throughput
MAX_LENGTH = 256  # Adjust based on text length

Model Caching

The model caches automatically in:

Linux/macOS: ~/.cache/huggingface/

Windows: %USERPROFILE%\.cache\huggingface\

🤝 Contributing

• Contributions are welcome! Please follow these steps:
• Fork the repository
• Create a feature branch (git checkout -b feature/amazing-feature) -Commit changes (git commit -m 'Add amazing feature') -Push to branch (git push origin feature/amazing-feature)
• Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

📚 Citation

If you use BhashaSetu in your research, please cite:

@software{bhashasetu2026,
  title = {BhashaSetu: Document-Level Machine Translation for Indic Languages},
  author = {T. Vishwanath, S. Samith, S. Vijay},
  year = {2026},
  url = {https://github.com/yourusername/bhashasetu}
}

@inproceedings{suryanarayanan-etal-2025-pralekha,
  title = "{PRALEKHA}: Cross-Lingual Document Alignment for {I}ndic Languages",
  author = "Suryanarayanan, Sanjay and Song, Haiyue and Khan, Mohammed Safi Ur Rahman and Kunchukuttan, Anoop and Dabre, Raj",
  booktitle = "Proceedings of IJCNLP-AACL 2025",
  year = "2025"
}

🙏 Acknowledgments

• AI4Bharat for IndicTrans2 model

• Pralekha Dataset creators

• Hugging Face for model hosting

• Streamlit for the amazing framework

📞 Contact

• T. Vishwanath - GitHub

• S. Samith - GitHub

• S. Vijay - GitHub

Project Link: https://github.com/Vishwanath-06/BhashaSethu

Built with ❤️ for Indic Languages | The Ethereal Conduit 🌉