Bard one

Back in February, I took a peek at ChatGPT, noticed at reference 1. Then towards the end of March, Google turned up with Bard. I kept the email, but did not do anything with it; it just never seemed to make it to the top of the to-do list. Then a few days ago, a correspondent tried it with an astronomical question – How many stars are there in the universe? – and I thought it did quite well. A handy summary of how you might go about answering such a question, together with a stab at the answer. Not stuff you should be trusting, but certainly stuff you could check with some combination of Google search, Bing search and Wikipedia - and from which you would probably learn. So clearly time for me to have a go, which I got around to a couple of days ago.

It so happens that I have been taking an interest in the sense of taste over the past few days, sweet, savoury (umami), bitter, salty, sour and all that sort of thing, so I thought I would start with that.

And Bard was quite good at giving me nicely written little summaries about various tasty matters. It responded quite well to probes. Sometimes it could say where it got its information from – an improvement on the expert systems of old which were, I believe, notoriously bad at explaining in plain language why they gave you the results that they did. Bard does rather better, giving you stuff that you can check.

There was some continuity in that you could refer to previous questions and answers and it would know what you were on about – although I dare say this only works within session.

It also responded to leading questions. So, for example, I asked if there were taste receptors for fat, to which it gave a clear answer of ‘no’, along with some useful tutorial material about what the body does know about fat. But when I probed a bit using ‘CF36’ as a clue, it seemed to dig around a bit more and, after a couple of goes, come up with a much more nuanced answer, much closer to the facts on the ground as I understand them. The catch there being that one had to know the clue first. 

With another possibility being that it worked out the answer that I wanted and, wanting to please, steered that way. Not so good.

It was quite good at general knowledge. It did rather well with the ‘pense-bête’ of a few days ago, for which see reference 4.

And sometimes it said no. I am only a language model and can’t handle that one.

But it did make some mistakes. And sometimes it invented stuff. Worse, sometimes you were not sure whether it had or not. Although, to be fair, there was a good supply of health warnings.

I think it did some inventing when, picking up on reference 1, I asked it about the Toller giant of references 5 and 6. Curiously, while it got the place – Eynesbury – right, it appeared to have made the rest of it up, despite enumerating various plausible looking local history sources. And it did not seem to be interested in taking on new information about the giant from me. Maybe one of the many strengths of Wikipedia is that it is very open in that way.

All in all, I was rather impressed. Many of the same strengths and weaknesses as ChatGPT, but I liked the way it had been packaged up and I thought it might well prove a useful tool. A useful supplement to search engines and Wikipedia.

In which connection it was interesting to read at reference 3 that Google senior management took the threat that ChatGPT posed to their search revenues very seriously indeed – ramping up their efforts with immediate effect. Action-this-day red flags all over the shop.

Other matters

Meanwhile, another correspondent had been playing with ChatGPT. He found it good at general knowledge and good on questions inviting waffle rather than answers. But useless when he asked it genuinely difficult questions, with one example being ‘what is the probability of their being a service denying outage at Amazon UK lasting more than 12 hours’. Apparently it did not even go as far as to enumerate the sort of supplementary questions that one would need to ask to tackle this one. Suggesting to me that, while ChatGPT might have hoovered up an awful lot of knowledge, it needs that knowledge to be organised, or at least to be organisable, in much the same way that knowledge is organised in a good text book. Such organised knowledge is not always going to be available for active, real world problems to which there is, as yet, no answer. From where I associated to Google’s success with protein folding: a computationally challenging business, but the rules of this game are known. And the game is closed away in its own little world.

Such matters no doubt occupy lots of powerful brains, real ones that is. Brains which worry about the two green boxes in the snap above, green boxes which reflect my own IT background in which there was a clear separation between data and process. A separation which I worry about from time to time: is it so clear, does it work at all, in the world of neural networks? And what about a separation of knowledge from language?

But at least I think that it is a rule of this particular game, that these systems built on large language models do not have specialist sub-systems, specialist add-ons. You can’t buy an add-on to do the history of science or the science of taste. Perhaps that will come.

A good by-product of all this was that I have learned that I can move text messages from my newish Samsung telephone to my oldish Microsoft flavoured laptop. The catch being that I can’t now do what I did then: I can still share a message with my gmail account but I can’t now work out how I converted a message to a pdf file and shared that. I should have been writing down what I was doing as I went. Another by-product was that I have been reminded that a text message is a much more primitive beast than an email message. Quick and easy, but with very little of the machinery that comes with an Outlook or gmail.

Conclusions

We shall see whether I get into the habit of using Bard.

References

Reference 1: https://psmv5.blogspot.com/2023/02/chatter.html.

Reference 2: https://bard.google.com/. 

Reference 3: https://en.wikipedia.org/wiki/Bard_(chatbot).

Reference 4: https://psmv5.blogspot.com/2023/06/its-snowing.html. 

Reference 5: https://psmv3.blogspot.com/2018/02/claim-to-fame.html. 

Reference 6: No tall story: just a giant's tale - Carol Moon – 1997.