Ready to pass the bar, not to practice law

The above headline could apply to most new law school grads.  (Not all new grads, of course; not all new grads can pass the bar.)  But I’m using the headline to describe an A.I. named Chat GPT.

A few years ago, I was convinced that A.I. had taken over legal tasks (if not entire legal jobs).  Not all tasks; you can’t send A.I. into the courtroom to try a case, for example.  But certainly it had taken over things like legal research and writing legal briefs, I thought.

And then I got an email from my state bar touting its seminar on how attorneys can put A.I., in the form of Chat GPT, to work for them.  I then researched it and learned that Chat GPT recently passed a bar exam.  Based on that, I signed up for Chat GPT and gave it a spin. 

I first asked the bot whether there are different burdens of proof in preliminary hearings in the different states.  It replied that, yes, there are different burdens of proof for such hearings, including preponderance of the evidence and probable cause.  I was surprised, as I had never heard of preponderance of the evidence for a prelim before, so I asked it which states use that burden of proof.  It responded that it was not aware of any states that use that burden at preliminary hearings.  What?!

I then threw it a softball.  I asked if yours truly had written any articles in law reviews.  It said that I had, and produced three examples.  The problem was that I hadn’t written any of the three that it produced.  I don’t think Mr. H. Patrick Furman would be pleased with the bot giving me credit for his work.  Nor is that a mistake I would expect from even a dumb bot, let alone one that passed a bar exam.  Pairing an article with its author seems like one of the easy tasks for A.I. to do—but then again, “Furman” and “Cicchini” are very similar names. 

Finally, I asked it to produce cases for me on a certain topic.  It produced cases, but the cases didn’t have anything to do with the subject matter.  It’s like I was reading the motion or brief of a sloppy prosecutor. 

This A.I. is no Cameron.  Maybe its day will come, but it’s not yet ready for primetime.