With South Africa preparing for the planned country-wide protests of the Economic Freedom Fighters (EFF) next week Monday 20th March, we decided to test out Microsoft’s much-touted Bing ChatGPT-powered chatbot with some very local questions.

The chatbot launched today on Microsoft Edge browsers and is available to anyone who used Edge, signs in on their Microsoft account and adds themselves to the Bing chatbot waitlist. Like ChatGPT, the bot uses trained artificial intelligence to produce human-like responses, and now with added Bing support, it scrapes the internet for search results which bolsters its answers.

Basically, it’s ChatGPT and Bing search all at once. When you first boot up a chat it will ask you what kind of responses you would like, whether more creative, more precise or more balanced. The creative options produce responses that are more original and imaginative, so we chose that one first.

First off, we asked Bing for the reasons why the EFF is launching the shutdown in the first place:

This allowed us to get a better understanding of how the chatbot works. It first checks for search results, and then compiles the results into a conversation like summary, complete with citations. It also makes sure to include relevant information, such as Julius Malema’s comments that the shutdown will make the 2021 July unrest look like “Micky Mouse.”

Next we asked it if any changes will come to South Africa via the EFF shutdown:

It changed the search query by itself to “EFF shutdown impact” and then listed a few changes South Africans could expect, using various online sources. Interestingly, one of the listed impacts took from three different sources, namely The Sowetan, Times Live and MSN. This is the believed impact on the economy, including the transport, mining, retail and education sectors.

Bing then offers a few more search queries for users to check out based on the questions you have already been asking:

We changed the answering type to “More Precise” and asked the impact question again, the answer changed quite a bit.

We appreciate that Bing gave a nuanced and measured answer here, and we agree with it for the most part. For the most part it reminds us of chatting with ChatGPT, in that it will try to provide balanced thought and often remark that it isn’t a human being so it can’t really weigh in with an opinion.

Next we tried to ask it a question in the same thread with directly mentioning the EFF, it didn’t disappoint and picked the topic right up.

We switched it “Balanced” to ask Bing what other recourses could the EFF take instead of launching a nationwide shutdown, and we think we may have found some of chatbot’s limitations here.

The first is that prompt crafting is likewise important for the Bing chatbot, like with ChatGPT. Using a prompt that is a bit unspecific can lead to unrelated answers, such as in the example above where it cites information from the US.

We switched it back to Creative as this seems to produce the best answers for more nuanced and specific questions, we fixed up the prompt a bit and asked it the same question again.

In all, we think that the ChatGPT-powered Bing is an incredibly powerful tool and one that will only get stronger and better with OpenAI’s continued improvements to its AI platform. It uses search results to provide citations to its answers, and while this seems to bolster responses it could also skew answers. If it grabs an article from a source that is not too objective, this could influence the answer.

We like the switching up between the three “answering modes” to create different kinds of responses. This could be a feature that one could play around with for hours to see what Bing spits out.

One thing we can confirm is that it definitely changes search, and makes the usual way of searching for information via Google-like website lists somewhat obsolete. If this is the future, advertisers will definitely be wanting to include paid-for answers that the chatbot gives.

Why would you look through article headlines when Bing can do it all for you? And in less time too.

We decided to ask Bing one last question about the EFF:

[Image – EFF on Twitter]