ChatGPT & SYSTRAN : Same technos, different objectives

ChatGPT and SYSTRAN use some of the same technologies, but with a different Goal in mind

LLM (Large Language Models)

This technology is not new. LLMs and translation models are mostly based on the same type of ‘transformers‘ algorithm.

The Transformers are the foundation for the SYSTRAN translation algorithms and models. We also use LLMs in our production processes (data cleanup, correction, etc.).

However, ChatGPT and GPT-4 have specialized their execution for generative AI and have communicated with the general public about it.

Machine learning

Language models and machine translation systems are driven from large amounts of text data. They use machine learning algorithms to identify patterns and structures in the training data. Once trained, these templates can be used to generate text or translate languages.

Pre-processing techniques

Language models and machine translation systems use preprocessing techniques to clean and normalize text data before using it for training.

Pre-trained language templates

Language templates are designed to generate text / predict the rest. ChatGPT is an LLM trained to answer questions. Machine translation systems can use pre-trained templates to speed training and improve translation accuracy.

Research: A Real Opportunity for SYSTRAN

SYSTRAN continues to lead NMT with its continued commitment to research and development of machine translation models. The company has a team of researchers and developers dedicated to creating innovative TCN models that are continuously tested and enhanced to deliver the best possible performance.

Systran also uses OpenAI technologies, the creator of ChatGPT , and in particular Whisper, speech processing.In fact, Whisper, for speech recognition and audio file processing, has been distributed as an OpenSource license, in accordance with the original OpenAI project and status.

Since then OpenAI has changed its status to become a for-profit company and strengthened its partnership with Microsoft. ChatGPT has not been opened except for descriptive paper, unlike previous OpenAI advances.

SYSTRAN contributes to open source through OpenNMT and continuously experiments with the latest open source LLMs, including ‘Bloom’ or other models.

University and community research around LLM also offers many opportunities for SYSTRAN. Managing training data and training transformation language models is essentially SYSTRAN’s main activity:

SYSTRAN already uses LLMs in its streams (e.g. BERT, T5, BLOOM, LABSE):

• In its data creation string: to align and filter sentences, to generate synthetic data, to classify / annotate data by domain, etc.
• More Experimental: to take advantage of monolingual data, to explore new use cases (bilingual writing aids, grammar correction, etc.)

SYSTRAN already uses language templates to improve the quality of its translation templates

As for performance, SYSTRAN already has state-of-the-art inference engines (running on CPUs), which deliver the best performance on the market for “reasonable” hardware conditions. This is based on the OpenNMT/Ctranslate2 project.

SYSTRAN is already exploring other quantification techniques (4-bit) to run these LLMs internally on smaller machines