Text Simplification with Deep Reinforcement Learning

Overview This repository contains an implementation of a neural text simplification model that combines sequence-to-sequence learning with reinforcement learning and lexical-semantic loss. The model aims to simplify complex text while maintaining meaning and grammatical correctness.

Architecture

The model consists of several key components: Encoder-Decoder with Attention: Uses LSTM layers with Bahdanau attention Reinforcement Learning Agent: Optimizes for SARI score Lexical-Semantic Loss: Maintains semantic meaning during simplification

Research Findings

Model Performance

1.Successfully combines supervised learning with RL

2.SARI score optimization through RL rewards

3.Effective semantic meaning preservation

Training Insights

1.Batch Size Impact

2.Larger batch sizes (128-256) show better convergence

3.Memory requirements increase significantly

Hardware Considerations

A100 GPUs provide optimal performance

H100 GPUs can reduce training time by 3-4x

Optimization Techniques

Mixed precision training

Gradient accumulation

Distributed training support

Performance Analysis

Training Metrics

Dataset: WikiLarge

Training pairs: 296,402

Validation pairs: 992

Test pairs: 359

Hardware Requirements For optimal performance:

GPU Model	VRAM	Batch Size	Training Time (50 epochs)
A100 80GB	80GB	256	~1 week- 2 weeks(minimum...)
A100 40GB	40GB	256	~ days
V100 32GB	32GB	128	~ days
T4 16GB	16GB	128	~ days
Advanced Hardware Options
For faster training:
GPU Model	VRAM	Batch Size	Est. Training Time
-----------	------	------------	-------------------
H100 80GB	80GB	512+	~ hours
8x H100	640GB	2048+	~ hours
8x A100	640GB	1024+	~ hours

You can use this architecture to create a Text simplifier on almost every sentence of english. (requires high GPUs for training).