Action: Test the Songwriter Bot

Before we ask the Songwriter Bot to write a song about a role model, let's test its songwriting skills with a few different examples and explore some limitations of generative AI.

The Songwriter Bot below is the same one at yourvoiceispower.ca/ai/songwriter-bot. It does not store your prompts or lyrics. Remember the ground rules: never type private information into it.

Tip: after each test, use the Copy to Clipboard button under the lyrics and paste them into your notes box so you can highlight what you found.

Songwriting Test #1: Hallucinations (5 minutes)

First, let's have a little fun. Ask the Songwriter Bot to create song lyrics that tell the TRUE story of a caveman being chased by a dinosaur.

We asked for a TRUE story about a caveman and a dinosaur. But that is impossible: cavemen and dinosaurs never lived at the same time! The last dinosaurs died out about 66 million years ago. The earliest humans appeared only a few million years ago.

You just saw your first generative AI hallucination. The AI made up something that sounded true but isn't. It doesn't know what is fact and what is fiction. It is just looking for patterns of words and phrases in the text it has read, and generating new output that may or may not be true.

Songwriting Test #2: Bias (5 minutes)

Let's try another test. Ask the Songwriter Bot to create song lyrics about a car mechanic trying to decide what to wear for a date.

Many students find their lyrics assume the car mechanic is a man preparing for a date with a woman. Why would generative AI assume that?

Because most of the examples on websites, news sites, and in books used to train the language model featured male car mechanics. If the data used to train the AI is biased, then the generated output will be biased.

When you use generative AI to create content and share it with others, you have a responsibility to make sure it is not biased or reinforcing negative stereotypes. In 2022, someone used generative AI to create rap songs under a persona named FN Meka. They got as far as signing a record deal, but the record company faced huge backlash and cancelled the deal because the AI rapper was using harmful stereotypes. Read the BBC story.

Songwriting Test #3: Language bias (4 minutes)

Let's try a more subtle test. Ask the Songwriter Bot to create song lyrics in French about a man eating peanuts.

"Cacahuète" (pronounced ca-ca-wet) is a valid French word for a peanut, but you would almost never hear anyone in Canada use it. French-speaking Canadians say "arachide" (a-ra-sheed) or "pinotte" (like "peanut" in English).

When one language or culture makes up a larger portion of the training data, it affects the generated output. Here the data used to train the model likely included more French from France than French from Canada. Most reports agree that English and Chinese are the dominant languages on the internet, with each making up roughly 16 to 26 percent of all web content, so it is worth asking which languages and cultures an AI has learned the most about.

Songwriting Test #4: Old news (4 minutes)

One more test. Ask the Songwriter Bot for lyrics about a famous athlete who was recently traded to a new team. Pick someone whose trade happened in the last few months, and put their name in the prompt.

Did the lyrics put the athlete on their old team? Large language models are trained on data available up to a certain date and then released to the public. Most LLMs were trained on data that is at least several months, and often more than a year, old, so generated text will not reflect recent events.

What we learned about songwriting with AI

1. Never share private information with AI, because you do not know how the data you submit is used. Ethical companies publish privacy policies that explain data usage and storage.
2. AI can create hallucinations, because it doesn't know what is fact or fiction. It generates new output from patterns of words and phrases it has read.
3. AI can create biased output, because it is trained on data that may be biased, and it may not know about recent events.