TeSum: Human-Generated Abstractive Summarization Corpus for Telugu
Authors : Ashok Urlana, Nirmal Surange, Pavan Baswani, Priyanka Ravva, Manish Shrivastava
About the dataset :
Expert human annotation for summarization is definitely an expensive task, and can not be done on huge scales. But with this work, we show that even with a crowd sourced summary generation approach, quality can be controlled by aggressive expert informed filtering and sampling-based human evaluation. We propose a pipeline that crowd-sources summarization data and then aggressively filters the content via: automatic and partial expert evaluation. Using this pipeline we create a high-quality Telugu Abstractive Summarization dataset (TeSum) which we validate with sampling-based human evaluation. We also provide baseline numbers for various models commonly used for summarization. A number of recently released datasets for summarization, scraped the web-content relying on the assumption that summary is made available with the article by the publishers. While this assumption holds for multiple resources (or news-sites) in English, it should not be generalised across languages without thorough analysis and verification. Our analysis clearly shows that this assumption does not hold true for most Indian language news resources. We show that our proposed filtration pipeline can even be applied to these large-scale scraped datasets to extract better quality article-summary pairs.
Corpus Statistics :
| Train | Validation | Test | ||||
|---|---|---|---|---|---|---|
| #Pairs | 16295 | 2017 | 2017 | |||
| Avg Compression% | 58.26 | 58.08 | 58.28 | |||
| Text | Summary | Text | Summary | Text | Summary | |
| #Unique Words | 183641 | 113723 | 49038 | 28873 | 49620 | 28777 |
| Avg Unique Words | 88.93 | 42.78 | 89.19 | 43.14 | 90.74 | 43.81 |
| (Min, Max) Words | (30, 536) | (12,213) | (32, 685) | (10, 248) | (36,592) | (12, 261) |
| Avg Words | 120.8 | 50.02 | 122.56 | 50.8 | 124.82 | 51.69 |
| Avg Sentences | 9.23 | 3.22 | 9.50 | 3.19 | 9.51 | 3.17 |
Download Dataset :
To download this dataset kindly fill the form given below.
If you use this corpus for your research, kindly cite it as follows:
Urlana, Ashok, et al. “TeSum: Human-Generated Abstractive Summarization Corpus for Telugu.” Proceedings of the Language Resources and Evaluation Conference, European Language Resources Association, 2022, pp. 5712–22. Language Technologies Research Centre, KCIS, IIIT Hyderabad. ltrc.iiit.ac.in/showfile.php?filename=downloads/teSum/