Sing it, Narrate it:
Quality Musical Lyrics Translation

To translate a paragraph of English musical lyrics into Chinese lyrics that both retain the original narrative role and are comfortable for singing, we focus on two kinds of constraints: translation quality and singability.

For translation quality, we seek to ensure that the translated lyrics are fluent, faithful to the original lyrics, and have literal beauty. For singability, we aim to ensure that the translated lyrics can be sung out naturally, which includes constraints on the length of the lyrics and the rhyme pattern. The following figure shows the constraints we consider in our work, with examples demonstrating their importance.

Our method consists of three key components: reward models trained to evaluate the quality of the translated language, a translation model trained using a two-stage pipeline which can generate sentence-level translations that are of high quality and satisfy singability constraints, and an inference-time optimization framework that composes sentence-level results into paragraph-level output. The following figure illustrates how these components work together.

We conduct extensive experiments including both automatic and human evaluations, where the results showcase the effectiveness of our method and demonstrate significant improvements over the baseline method. Here we provide more demos along with their audios.