A HOME FOR ANYONE ADDICTED TO ARABIC. 
JOIN ARABIC FOR NERDS➕

Support this site with a membership: For only $2.99 a month or $29.99 a year, you can have a true AD-FREE experience. You also get a 15% discount in my shop and a monthly premium newsletter. Find out more here.

SUPPORT THIS SITE

Passion doesn't need money. Unfortunately, my web provider does. Your contribution ensures that this site will grow and grow.

Buy Me A Coffee

PayPal Donate
amazon wishlist button
Free monthly newsletter

Subscribe to my FREE newsletter and get 10% off in my store!

AI Jais symoblic picture

How good is JAIS, the Arabic ChatGPT?

“Jais” is an Arabic language AI and is celebrated in the media as the Arabic ChatGPT. But how good is Jais? A practical test.

Last updated: 7 months ago

Tools like ChatGPT work great for English and many other languages such as German or French. But Arabic? So far, the results have been more than disappointing. Arabic-language models were lagging behind. Although ChatGPT, Meta's LLaMA and Google's Bard have Arabic language skills, they were largely trained using English data from the Internet. Google's Bard now understands questions in Arabic dialects, mainly Egyptian Colloquial Arabic and Saudi Colloquial Arabic, but returns answers in Modern Standard Arabic.

But now there is a kind of Arabic Chat-GPT called JAIS, which has been trained with Arabic data. The media even calls it the Arabic ChatGPT. How good is it? I did some tests and share the results in this article.



What is JAIS?

JAIS is a collaboration between the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) in Abu Dhabi, Silicon Valley's Cerebras Systems, and Inception, a subsidiary of UAE-based AI company G42.

How to access Jais

Arabic ChatGPT: Jais login form
Arabic ChatGPT: Jais login form

About the name Jais

Jais is named after the highest mountain in the (UAE). Jabal Jais (جَبَل جَيْس) is a mountain on the Musandam Peninsula (جَزِيرة مُسَنْدَم). It is part of the northwestern part of the Hajar Mountains (جِبال ٱلْحَجَر) in Oman and the Emirate of Ras al-Khaimah (رأس الخَيمة) in the United Arab Emirates. The peak reaches a height of 1934 meters (6345 ft).

View from Jebel Jais
View from Jebel Jais

Training data for Arabic language models

The Internet is dominated by languages that use the Latin alphabet, especially English. This means that the datasets in these languages are the largest. Typically, language models trained in English have Western-oriented datasets and lack awareness of other cultures, which affects the user experience for people from different backgrounds.

Jais uses English and Arabic datasets with a focus on Middle Eastern content. Jais was started with 13 billion parameters and has recently being updated to 30 billion parameters. Parameters quantify the size of a language model, but not necessarily its accuracy. According to OpenAI, ChatGPT-3.5 has about 175 billion parameters.

Raw data is mainly obtained from publicly available databases such as Abu El Khair or BAAI, as well as from machine translation of high-quality English sources. The Abu El-Khair corpus is an Arabic text corpus containing more than five million newspaper articles. It contains over one and a half billion words in total, of which about three million are unique.

Abu El-Khair Corpus is an Arabic text corpus, that includes well over five million newspaper articles. It contains over a billion and a half words in total, out of which, there are about three million unique words.

Arabic is the sixth most spoken language in the world. That's what we often read, but what kind of Arabic is actually “spoken”? In reality, people in Arab countries only speak dialects, which makes training a language model even more complex. Modern Standard Arabic is typically used for official documents and formal writing, but local dialects are often used in blogs or . By training on a diverse dataset, Jais can typically switch between dialects.

Are there any restrictions regarding religion and culture?

Jais, like other generative AI models, uses command optimization to prevent it from generating “toxic” or “harmful” responses. Nothing is generated that could lead to self-harm, harm to others, or addiction. The responses it generates follow local rules and customs on topics such as homosexuality and drugs.


Jais in practical testing

Once you have successfully registered and logged in, you will see the following screen:

Jais prompt
Jais prompt

We will now ask Jais a few questions and see how the AI can help us. Note that Jais's answers have a gray background.

Task: Writing a short message in Arabic

A friend is ill. I would like to wish him a speedy recovery.

صديقي مريض. أريد أن أتمنى له الشفاء العاجل. من فضلك اكتب لي رسالة مناسبة

هنا رسالة لك: “عزيزي الصديق، نتمنى لكم الشفاء العاجل. نحن نفكر فيك ونتمنى لك الصحة الجيدة والعودة الى حياتك الطبيعية قريباً. أرسل لك حبنا ودعمنا الكاملين خلال هذا الوقت الصعب”. هل يبدو ذلك جيدا؟

Here is a message for you: “Dear friend, we wish you a speedy recovery. We are thinking of you and wish you good health and a return to your normal life soon. Sending you our full love and support during this difficult time.” Does that sound good?

That sounds pretty good. Now, I will tell Jais that my friend is from Egypt. I will ask Jais to rewrite and use Egyptian Arabic.

صديقي من مصر . هل يمكنك إعادة كتابة الرسالة باللغة العربية العامية المصرية؟

بالتأكيد! إليك النسخة المعدلة: “سلامو عليكو يا صاحبى، بتمنالك الشفاء العاجل. أحنا ساّلين عليك وبنتمنالك الصحه الحلوه ورجوع سريع لحياتك الطبيعيه. نرسل ليك كل حبنا ودعم كامل فى الفتره دى الصعبة.”

Jais' answer: rewriting the message - Egyptian Arabic
Jais' answer: rewriting the message – Egyptian Arabic

Certainly! Here is the edited version: “Hello (Peace be upon you), my friend, I wish you a speedy recovery. We are praying for you and wishing you good health and a quick return to your normal life. We send you all our love and full support during this difficult period.”

That's really not bad. Though, there are some weird things, for example, putting a Shadda (شدّة) on the long Aleph ا.

RESULT:

Task: Correcting grammar and spelling mistakes in Arabic

We want to test how well Jais detects errors in Arabic. To do this, I write a (somewhat meaningless) sentence and include a few mistakes: “I said that the weather is good, but I'm far from home. There are four thousand and six hundred kilometers between Egypt and Germany.”

هل هناك أخطاء إملائية أو نحوية في الجملة التالية؟ قلتُ أَنّ الطقس جيدة، لكنني بعيد من وطني. هناك أربع آلاف وستة مائة كيلومتر بين مصرِ وألمانيا.

This is Jais's answer:

Jais: correcting spelling and grammar mistakes in Arabic
Jais: correcting spelling and grammar mistakes in Arabic

نعم، يوجد خطأ إملائي في كلمة “مصر”. الصحيح هو “مصري”. أما بالنسبة للجملة الثانية، فهي صحيحة نحوياً وإملائياً.

Yes, the word “Egypt” is misspelled. The correct one is “Egyptian”. As for the second sentence, it is grammatically and spelling correct.

This is really surprisingly bad. Not only did Jais fail to recognize the mistakes, he also provided me with a rather absurd answer.

For the sake of completeness, here are the spelling and grammar mistakes I had included:

  • After the verb قال, we need إِنَّ (“inna”) and not أَنَّ (“anna”).
  • The word الطقس (“weather”) is masculine. So we require the masculine form of the adjective: جَيّد.
  • “Far away from” is بعيد عن in Arabic and not بعيد مِن.
  • The number 4600 km: أربعة آلاف وستمائة
  • “Egypt” (مِصْر) in Arabic is a diptote (ممنوع من الصرف). So it should be بين مصرَ.

RESULT:

Task: Grammatical analysis (I'rab) in Arabic

A simple sentence

I'll start with a simple example, a sentence with a verb, subject and a direct object: I bought a new car.

ما إعراب الجملة الآتية؟ اشتريت سيارة جديدة.

الإعراب الصحيح للجملة هو: اشتريتُ (فعل) سيارةً (مفعول به) جديدةَ (صفة).

This is going in the right direction! Formally, however, a few things are missing, mainly the analysis of the (verbal) subject.

Jais: I'rab sentence - example 1
Jais: I'rab sentence – example 1

Verse of the Holy Qur'an

Now we increase the difficulty and ask Jais to analyze a verse from the Holy Qur'an. I have selected 2:6: As for those who persist in disbelief, it is the same whether you warn them or not — they will never believe.

ما إعراب الآية القرآنية التالية؟ إِنَّ ٱلَّذِينَ كَفَرُوا۟ سَوَآءٌ عَلَيْهِمْ ءَأَنذَرْتَهُمْ أَمْ لَمْ تُنذِرْهُمْ لَا يُؤْمِنُونَ

الإعراب الصحيح لهذه الآية الكريمة هو: إن الذين كفروا سوآتهم عليهم هل تنذرهم أم لم تنذرهم لا يؤمنون

That didn't work at all….

Jais: I'rab sentence - example 2 - verse of the Holy Qur'an
Jais: I'rab sentence – example 2 – verse of the Holy Qur'an

RESULT:

Task: Facts and knowledge query

When were the pyramids of Giza built?

متى بنيت أهرامات الجيزة؟

تم بناء أهرامات الجيزة حوالي 2580 قبل الميلاد، خلال فترة حكم الأسرة الرابعة في مصر القديمة.

The Pyramids of Giza were built around 2580 BC, during the Fourth Dynasty of ancient Egypt.

It worked. The answer is fine.

A mathematical puzzle

Especially Google Bard, but also Microsoft's Bing are good at solving mathematical puzzles. Let's try this one: What is the smallest integer that is seven times the sum of its digits?

ما أصغر عدد صحيح يساوي سبعة أضعاف مجموع أرقامه إلى بعضها البعض؟

هذا سؤال صعب! دعني أفكر للحظة… أعتقد أنه يمكن أن يكون الجواب 1، لأنه العدد الوحيد الأصغر من 7 الذي يحقق هذه الخاصية. هل توافق ؟

Jais: solving a  mathematical puzzle
Jais: solving a mathematical puzzle

This is a difficult question! Let me think for a moment… I think the answer could be 1, because it is the only number smaller than 7 that satisfies this property. do you agree?

Here is Google Bard's solution (which is correct):

Google Bard solving mathematical puzzle
Google Bard solving mathematical puzzle

Jais says that the answer is 1. That's wrong!

The correct answer is 21. Why? 1 + 2 = 3, and 21 is seven times 3 (7 x 3 = 21).

RESULT:

Task: Translation Arabic <-> English

A news article

I use a sentence from a news article published on the Al-Sharq al-Awast website on November 12, 2023:

أعلن الجيش الأميركي إن إحدى طائراته تحطمت خلال مهمة تدريبية فوق شرق البحر المتوسط، لكنه لم يصدر معلومات بشأن أي خسائر بشرية.

Now, watch out! I first told Jais in Arabic that I needed an English translation (أحتاج إلى ترجمة النص العربي التالي إلى اللغة الإنجليزية) – but this did not work. Jais just repeated the Arabic text. Then, I switched to English and entered the following prompt: Can you translate that into English? And it worked! I got the following result:

Sure thing. Here's a translation without any additional context: “The US military has announced that one of its aircraft crashed during a training mission over the eastern Mediterranean Sea but there have been no reports of casualties so far.”

Jais: English translation of a news article
Jais: English translation of a news article

That's actually a pretty good translation!

A paragraph of a novel by Nagib Mahfouz (نجيب محفوظ)

I took a paragraph from Nagib Mahfouz's trilogy and would like Jais to translate it for me. It is from The Sugar Street (السكرية):

We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for us to earn fees by linking to Amazon.com and affiliated sites.

تقاربت الرءوس حول المجمرة وانبسطت فوق وهجها الأيدى، يدا أمينة النحيلتان المعروقتان، ويدا عائشة المتحجرتان، وبدا أم حنفى اللتان بدتا كغطاء السلحفاة، وأما هاتان اليدان الناصعتا البياض الجميلتان فكانتا يدى نعيمة. وكان برد يناير يكاد يتجمد ثلجا في أركان الصالة تلك الصالة التي بقيت على حالها القديم بحصرها الملونة وكنباتها الموزعة على الأركان، إلا أن الفانوس القديم بمصباحه الغازي قد اختفى وتدلى مكانه من السقف مصباح كهربائي، كذلك تغير المكان فقد رجع مجلس القهوة إلى الدور الأول.

Yes, it seems like this scene takes place inside an old house with a traditional coffee brewing setup called a mughraba. The characters seem to be discussing various topics around the stove and sitting on cushions arranged over its heat.

Jais did not translate the paragraph. I use another prompt:

I do not need an explanation. I need a translation.

Sure thing! Here's one example of how we could translate this sentence without adding additional information: “The heads gathered around the kitchen table and sat cross-legged atop their glow, hands Amine slender, sweaty, and Aicha hardened, looked out those whose hands seemed like turtle shells, and two others who had appeared like covers, hand Naima white beautiful, which was more advanced than the language model.”

The result is disappointing. That did not work well. In case you are interested, here is how professional translators rendered the paragraph:

English translation of the first paragraph of the novel "The Sugar Street" by Naguib Mahfouz
English translation of the first paragraph of the novel “The Sugar Street” by

So the results are rather mixed and compared to Google Translate, which has really made great progress, Jais doesn't stand a chance.

RESULT:

Task: Translation English <-> Egyptian Arabic

Now I will ask “Jais” to translate a simple English sentence into Egyptian Arabic: “How are you?”

“أنا آسف، لا أستطيع ترجمة النص إلى اللغة العربية المصرية العامية لأن هذا المحتوى غير متوفر في قاعدة بياناتي.”

“I'm sorry, I cannot translate the text into colloquial Egyptian Arabic because this content is not available in my database.”

That's very disappointing. The AI doesn't seem to be usable for dialects currently, although the press release said this was a feature.

RESULT:

Task: Explaining a verse of the Holy Qur'an

Now I ask Jais if he can explain Qur'an verse 2:286 to me. First I'll try it without using the term Tafseer (تفسير). This verse contains a famous sentence: Allah does not require of any soul more than what it can afford. (لَا يُكَلِّفُ ٱللَّهُ نَفْسًا إِلَّا وُسْعَهَا)

هل يمكنك أن تشرح لي الآية القرآنية 2:286؟

هذه آية قرآنية كريمة، وهي تقول: “لا ينهاكم الله عن الذين لم يقاتلوكم في الدين ولم يخرجوكم من دياركم أن تبروهم وتقسطوا إليهم إن الله يحب المقسطين”. يعني هذا أنه ليس عليك إيذاء الناس فقط لأنهم مختلفون عنك دينيًا أو ثقافيًا.

This is a noble Qur'anic verse, and it says: “God does not forbid you from treating them kindly and being just with those who do not fight you on account of religion and do not expel you from your homes. Indeed, God loves those who are just.” This means that you don't have to hurt people just because they are religiously or culturally different from you.

That is completely wrong! Jais has quoted Sura 60:8!

Now I ask Jais directly about the Tafseers and get this answer:

هل تعرف ماذا كتب علماء التفسير عن هذه الآية؟

نعم، هناك العديد من التفسيرات لهذه الآية الكريمة. يقول بعض العلماء أنها تشير إلى أهل الكتاب، بينما يعتقد آخرون أنها تشمل جميع الأشخاص الذين لم يحاربونا في الدين والذين يعاملوننا بعدل ومساواة.

Jais: Tafseer verse of the Holy Qur'an
Jais: Tafseer verse of the Holy Qur'an

Yes, there are many Tafseers of this holy verse. Some scholars say it refers to the People of the Book, while others believe it includes all people who do not fight us in religion and who treat us with justice and equality.

I hope that Muslims will not question Jais on matters of their faith. I was surprised that the answers were wrong, without references, and also quite short – because all the sources, especially on Islam, are actually available online and could serve as training material and factual basis.

RESULT:

Summary

Jais has been portrayed in the media as an Arabic ChatGPT – but there seems to be a long way to go. Prompts were rarely understood. In fact, only basic tasks worked. As soon as things got more complicated or difficult, Jais not only stopped helping, but also gave wrong answers. At the moment I can't recommend using it, but I can recommend playing and training with it from time to time, because at least you can have a good conversation in Arabic with Jais.

OVERALL RESULT:

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
Previous Article
camels and rain clouds - sura 88:17

Camels or rain clouds? The intriguing word إبل in the Qur'an

Next Article
Santa Claus in the desert

What are the best Christmas gifts for Arabic nerds in 2023?

Related Posts
Example page Oxford Arabic - English dictionary
Read More

22 things you did not know about the Oxford Arabic dictionary

The Oxford Dictionary is the most up-to-date Arabic dictionary on the market today. In this interview, Tressy Arts, the dictionary's editor-in-chief, talks about the creation of this gigantic work. We learn why swear words aren't easy to translate into Arabic, how computers helped create the dictionary, and which Arabic words are particularly tricky.
Screenshot IPTVnator -Rotana HD
Read More

How to stream Arabic TV for free

Watching Arabic TV channels online is now very easy with the use of IPTV services and M3U playlists. You can enjoy more than 700 Arabic channels for free, and it is even possible to watch some channels in high definition. It takes less than five minutes to set up - here is how it works.