Freshly back at my desk after a week of wandering around the countryside and swimming in the sea, the thought occurred, “Do I really need to be here? Why haven’t I been replaced by a robot yet?”.

After all, we have been promised that artificial general intelligence (AGI) is around the corner for some years now, and at large scale fewer humans being needed to do the same amount of work seems inevitable. But could it do my job? I decided, for just one morning, to note down everything I did in my current role managing the IT dept of a small charity in the education sector and to ask the question “Could a robot have done this?”. In each case I would consider the answer in some (some would say tedious) detail, to try and get a sense of how round-the-corner the new utopia is.

9.00 – Catching up on emails, responding quickly to easy queries, deleting salesy ones, leaving the those that need a more considered response.

Of course I’m only seeing a fraction of what is emailed to me because the majority of the spam sent to me is pre-filtered by my email provider. Some of the easy queries could perhaps have been responded to by a bot, but if so the sender could have simply consulted a bot themselves rather than emailing me. Ergo we can suppose this task, as long as it exists, will need the human touch.

Humans 1, robots 0

9.38 – A colleague needs help with a workflow built into a proprietary application

Were this a problem with Power Automate, Co-Pilot would have easily volunteered a solution. However, in this case the software is unknown to internet-trained generative AI, which would only be able to help at a high level, and not the detailed step-by-step required by my colleague.

You could argue that this software could one day have its own AI assistant. However this is unlikely if the software is very bespoke, unless AI becomes much, much cheaper. Lets award a point to both humans and robots for this one.

Humans 2, robots 1

9.48 – An external API (Hubspot) has been updated requiring a change to a power query that is part of a symantic model in Power BI.

In order for a robot to do this it would have to

1. Read and interpret the email from Hubspot.
2. Know that this particular API is in use by a symantic model.
3. Find and read the necessary API documentation.
4. Find and open the file containing the symantic model.
5. Identify and implement a solution to the power query
6. Update the Power BI service and test.

Each of these steps seems to me well within the capabilities of AI. (Confession: AI helped me write the power query in the first place). What’s missing here is all encompassing bot that can read the email and has access to the symantic models in the Power BI service. This would have to be an AI that has access to all the cloud services necessary (email, Hubspot, Power BI) and be given the permissions necessary to make changes. In principle though, there was/is no subtask here that couldn’t be done by a bot.

So… Humans 2, Robots 2.

Conclusion

It appears there are still plenty of occasions when only a human will do, even in an ostensibly technical role. In just one morning I identified the following features where AI currently cannot replace human intervention:

1. Overall oversight. AI can read emails, research online documents and write code. But it still needs a director prompting it to do each of these subtasks and tying them together.
2. Initiative. AI with overall oversight would still need to have initiative to do anything with that oversight.
3. Permissions. AI with overall oversight and initiative will still need permission to make changes which relates to…
4. Responsibility.
5. Dealing with novelty. In common situations AI excels. For example Co-Pilot will create a complex excel formula quicker and better than any human given the right prompts. But with novel, bespoke software, it simply hasn’t had enough material to learn from.

The case is the multi-select, multiple choice question, where a learner may select x number of correct answers from y number of options.

The client particularly did not want to divulge how many correct answers there were, so out of 8 options, the learner could choose anything up to 8 answers. The client’s point of view was that telling the learner how many answers were correct was too much of a clue, and that part of what they were seeking to test was the learners’ ability to confidently distinguish the true statements from the false, rather than just guess the most plausible.

However, this can have large implications.

If we say “Choose the correct 2 options from the 8 options listed”, the learner must always choose 2 options. This means the total possible number of responses are:

8!/2!⋅(8−2)!​=28

So there are 28 ways to answer this question.

But if we don’t tell the learner there are 2 correct options they could choose 1, 2, or anything up to all 8. The total number possible responses is the total of the possible combinations. So…

8+28+56+70+56+28+8+1=255

Depending on how the marking rubric is stored, this could make quite a difference. A flat table of possible responses would require 255 rows.

Where it gets even more complicated is when we want to give partial scores for partially correct answers. Lets say the learner gets 2 marks for each correct selection. 4 marks in total are possible. But what if they choose the 2 correct options and a third incorrect one? What if they choose six? It is not obvious how such responses would be scored.

I posed this question to Co-Pilot and it simply recommended specifying the correct number of options to minimize these difficulties. So it seems there are no easy answers here!

I typed this into Co-Pilot:

Given an average rating out of 5 and a count of the number of ratings, and assuming every good review is 5 and every bad review is 1, create a function in excel to calculate the number of bad reviews

Co-pilot’s response:

Let’s assume:
A1 contains the average rating.
B1 contains the total number of ratings.
The formula to calculate the number of bad reviews (where each bad review is rated as 1) is:
= (5*B1 – A1*B1) / 4

Here’s the breakdown of the formula:
5*B1 calculates the total points if all reviews were good (rated 5).
A1*B1 calculates the total points actually received.
The difference (5*B1 – A1*B1) gives the total points lost due to bad reviews.
Dividing by 4 gives the number of bad reviews because each bad review contributes 4 points less than a good review (5-1=4).

Pretty clever!