Parachute: Evaluating Interactive Human-LM Co-writing Systems
Informations
- Type:
- misc
- Auteurs:
- Hua Shen and Tongshuang Wu
- Pertinence:
-
Haute
- Référence:
- Doi:
- 10.1609/aiide.v18i1.21955
- Mots-clés:
- Url:
- https://arxiv.org/abs/2303.06333
- Date de publication:
- 03/2023
- Résumé:
- framework pour évaluer un outil de co-writing avec un llm
- Abstract:
- A surge of advances in language models (LMs) has led to significant interest in using LMs to build co-writing systems, in which humans and LMs interactively contribute to a shared writing artifact. However, there is a lack of studies assessing co-writing systems in interactive settings. We propose a human-centered evaluation framework, Parachute, for interactive co-writing systems. Parachute showcases an integrative view of interaction evaluation, where each evaluation aspect consists of categorized practical metrics. Furthermore, we present Parachute with a use case to demonstrate how to evaluate and compare co-writing systems using Parachute.
- Pdf:
- Lien pdf
Références
1 articles
| Titre | Type | Pertinence | Auteurs | Date Publication | Références | Citations | Actions |
|---|---|---|---|---|---|---|---|
| Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence | article | Haute | Singh, Nikhil and Bernal, Guillermo and Savchenko, Daria and Glassman, Elena L. | 02/2022 | 0 | 4 |
Citations
0 articles
| Titre | Type | Pertinence | Auteurs | Date Publication | Références | Citations | Actions |
|---|---|---|---|---|---|---|---|
| Pas encore d'article | |||||||
Mots-clés
0 mots-clés
| Nom | Nombre d'articles | Actions |
|---|---|---|
| Pas encore de mot-clé | ||
Auteurs
1 auteurs
| Nom | Nombre d'articles | Actions |
|---|---|---|
| Hua Shen and Tongshuang Wu | 1 |