December 4, 2024

Using OpenAI's Function Calling

Function Calling is one of the most impactful mechanisms I’ve encountered so far when it comes to enhancing the user experience when working with LLMs. OpenAI provides a way to define a set of functions the model can access (referred to as “tools”). The model then determines when to use any of these tools based on the context. It’s certainly worth exploring for a more seamless user experience.

A bit of context

For me Function Calling is a powerful way to integrate an LLM with custom logic that you have developed (it can be an API, a set of helpers, etc.). In this paradigm you have two players: the LLM model and the set of functions (tools) available to your model.

In this article I'll introduce my experience using the OpenAI's Function Calling API.

The experiment

I created a travel assistant bot that specializes in finding cheap flights and hotels.

To achieve this, I provided the bot with access to a set of tools designed to streamline the search for flights and hotels, ensuring it delivers better results to users.

Since the steps to configure a bot that calls functions is properly explained in the OpenAI guide, I won't cover it here to avoid repetition. Instead, I'll just introduce my code and results.

The bot

I created a new OpenAI chat completion client, using the "gpt-4o" model. The client initialization is pretty straightforward with the only difference that it will now accept a new "tools" attribute. This attribute is the list of all available functions (tools) that your bot will have access to. Initializing this set of tools requires following a predefined schema in which you will specify the name of the function, its description, and the parameters that it requires. Particularly the description is very important since it will train the model about how and under what circumstances use the tool.

Okay, enough talk, here is the code of my bot (the code of the tools will come after):

1require('dotenv').config();
2import { input } from '@inquirer/prompts';
3import OpenAI from "openai";
4
5import { tools, functionsMap } from "./blogFunctions";
6
7const apiKey = process.env.OPENAI_API_KEY;
8const openai = new OpenAI({
9  apiKey,
10});
11
12async function main() {
13  const initialAssistantMessage = 'What can I help with?';
14  let prompt = await input({ message: initialAssistantMessage });
15
16  const messages: OpenAI.ChatCompletionMessageParam[] = [
17    { role: "system", content: "You are a helpful travel expert assistant. You have vast knowledge of airlines, destinations, cheap air fares, and good hotels. Use the supplied tools to assist the user." },
18    { role: "system", content: "You are responsible for assisting in requirements related to planning trips, specifically finding cheap flights and hotels." },
19    { role: "system", content: "The supplied tools will give you unlimited access to flights between destinations and hotels. Generate your response using plain text." },
20    { role: "system", content: "If (and only if) you have finished helping the user you can politely suggest them to exit the session by typing the word 'exit'." },
21    { role: "assistant", content: initialAssistantMessage },
22    { role: "user", content: prompt },
23  ];
24
25  let isFunctionCallTurn = false;
26  while (prompt !== 'exit') {
27    if (isFunctionCallTurn) {
28      console.log('[Function call]');
29    } else {
30      console.log('...');
31    }
32    const response = await openai.chat.completions.create({
33      model: "gpt-4o",
34      messages: messages,
35      tools: tools,
36      parallel_tool_calls: false, // set to true if you want to make multiple tool calls in parallel
37    });
38
39    if (response) {
40      // check if the assistant is calling a function
41      if (response.choices[0].message.tool_calls?.length) {
42        const toolCall = response.choices[0].message.tool_calls?.[0];
43        const functionName = toolCall.function.name;
44        const functionArguments = JSON.parse(toolCall.function.arguments);
45
46        // get the function from the map
47        const fn = functionsMap.get(functionName);
48        if (fn) {
49          console.log(`[${functionName}]`);
50          isFunctionCallTurn = true;
51          const functionCallResult = await fn(functionArguments);
52          const functionCallResultMessage: OpenAI.ChatCompletionToolMessageParam = {
53            role: "tool",
54            content: JSON.stringify({
55              ...functionArguments,
56              result: functionCallResult,
57            }),
58            tool_call_id: toolCall.id,
59          };
60
61          // add the tool call and result to the messages
62          messages.push(response.choices[0].message);
63          messages.push(functionCallResultMessage);
64        }
65      } else {
66        const content = response.choices[0].message.content;
67        console.log(`Assistant: ${content}`);
68        prompt = await input({ message: 'You:' });
69        messages.push({ role: "assistant", content });
70        messages.push({ role: "user", content: prompt });
71
72        isFunctionCallTurn = false;
73      }
74    }
75  }
76
77  console.log('Goodbye!');
78};
79
80main();
81

The functions (tools)

As you may have noticed, we need to define the list of “tools” that the bot expects. To keep the code cleaner, I placed these functions in a separate file.

For the purposes of this experiment, I created a couple of mock functions that return fake flight and hotel information. Additionally, in this file, I defined OpenAI’s list of tools, where, as mentioned earlier, we specify relevant details about the functions. If you’re using TypeScript, you can take advantage of the OpenAI.ChatCompletionTool[] definition for a more robust and safer development experience.

Here is the code of my functions:

1import { faker } from '@faker-js/faker';
2import OpenAI from "openai";
3
4const delay = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));
5const getRandomDelay = () => Math.floor(Math.random() * 5000) + 1000;
6
7const getFlightsBetweenCities = async ({ origin, destination }: any) => {
8  await delay(getRandomDelay());
9  const flights: any = [];
10  for (let i = 0; i < 6; i ++) {
11    const airlines = ["Delta", "American Airlines", "United", "Southwest", "JetBlue", "Alaska"];
12    flights.push({
13      flightNumber: faker.airline.flightNumber(),
14      aircraft: faker.airline.aircraftType(),
15      airline: airlines[Math.floor(Math.random() * airlines.length)],
16      origin,
17      destination,
18      departureTime: faker.date.recent(),
19      arrivalTime: faker.date.future(),
20      price: faker.commerce.price(),
21    });
22  }
23
24  return flights;
25};
26
27const getHotels = async ({ date, city }: any) => {
28  await delay(getRandomDelay());
29  const hotels: any = [];
30  for (let i = 0; i < 6; i ++) {
31    hotels.push({
32      name: faker.company.name(),
33      location: city,
34      checkIn: faker.date.recent(),
35      checkOut: faker.date.future(),
36      price: faker.commerce.price(),
37    });
38  }
39
40  return hotels;
41};
42
43const tools: OpenAI.ChatCompletionTool[] = [
44  {
45    type: "function",
46    function: {
47      name: "getFlightsBetweenCities",
48      description: "Get a list of flights between two cities. Call this function when the user asks for flights between two cities, or when they ask for cheap flights between two locations.",
49      parameters: {
50        type: "object",
51        properties: {
52          origin: {
53            type: "string",
54            description: "The origin city of the flight",
55          },
56          destination: {
57            type: "string",
58            description: "The destination city of the flight",
59          },
60        },
61        required: ["origin", "destination"],
62        additionalProperties: false,
63      },
64      strict: true,
65    },
66  },
67  {
68    type: "function",
69    function: {
70      name: "getHotels",
71      description: "Get a list of hotels in a city. Call this function when the user asks for hotels in a specific city, or when they ask for cheap hotels in a location.",
72      parameters: {
73        type: "object",
74        properties: {
75          city: {
76            type: "string",
77            description: "The city where the hotels are located",
78          },
79          date: {
80            type: "string",
81            description: "The date for which the user wants to book the hotel",
82          },
83        },
84        required: ["city", "date"],
85        additionalProperties: false,
86      },
87      strict: true,
88    },
89  }
90];
91
92const functionsMap = new Map<string, Function>();
93functionsMap.set('getFlightsBetweenCities', getFlightsBetweenCities);
94functionsMap.set('getHotels', getHotels);
95
96export {
97  tools,
98  functionsMap,
99};
100

The result

After building and running the bot, here is a quick example of the results!

1node ./dist/function_calling/blogBot.js
2✔ What can I help with? i need to find a cheap flight to miami
3...
4Assistant: To help you find a cheap flight to Miami, could you please let me know your departure city?
5✔ You: winnipeg
6...
7[getFlightsBetweenCities]
8[Function call]
9Assistant: Here are some flight options from Winnipeg to Miami:
10
111. **JetBlue**
12   - **Flight Number:** 6
13   - **Departure:** 2024-12-03, 16:53
14   - **Arrival:** 2025-04-24, 01:52
15   - **Price:** $383.75
16   - **Aircraft:** Widebody
17
182. **Alaska Airlines**
19   - **Flight Number:** 75
20   - **Departure:** 2024-12-04, 00:05
21   - **Arrival:** 2025-03-21, 09:37
22   - **Price:** $88.79 (This is the cheapest option)
23   - **Aircraft:** Widebody
24
253. **Delta**
26   - **Flight Number:** 4310
27   - **Departure:** 2024-12-03, 15:02
28   - **Arrival:** 2025-11-27, 08:15
29   - **Price:** $937.89
30   - **Aircraft:** Regional
31
324. **United Airlines**
33   - **Flight Number:** 46
34   - **Departure:** 2024-12-03, 22:10
35   - **Arrival:** 2025-02-18, 19:09
36   - **Price:** $90.85
37   - **Aircraft:** Widebody
38
395. **Alaska Airlines**
40   - **Flight Number:** 28
41   - **Departure:** 2024-12-04, 02:22
42   - **Arrival:** 2025-05-17, 22:59
43   - **Price:** $784.19
44   - **Aircraft:** Narrowbody
45
466. **Alaska Airlines**
47   - **Flight Number:** 84
48   - **Departure:** 2024-12-03, 21:14
49   - **Arrival:** 2025-04-08, 00:55
50   - **Price:** $558.05
51   - **Aircraft:** Regional
52
53The cheapest option is with Alaska Airlines, flight number 75, priced at $88.79. Let me know if you would like assistance with anything else!
54✔ You: i need to find somewhere to stay
55...
56Assistant: What date are you planning to check in, so I can find you the best hotel options in Miami?
57✔ You: december 23
58...
59[getHotels]
60[Function call]
61Assistant: Here are some hotel options in Miami for your check-in date on December 23, 2024:
62
631. **Gibson, Lueilwitz, and Batz**
64   - **Price:** $118.95
65   - **Available from:** December 3, 2024
66   - **Available until:** July 11, 2025
67
682. **Bradtke Group**
69   - **Price:** $455.79
70   - **Available from:** December 4, 2024
71   - **Available until:** February 17, 2025
72
733. **McLaughlin, Lubowitz, and King**
74   - **Price:** $447.15
75   - **Available from:** December 3, 2024
76   - **Available until:** February 7, 2025
77
784. **Thompson, Mills, and Schaden**
79   - **Price:** $751.09
80   - **Available from:** December 3, 2024
81   - **Available until:** December 31, 2024
82
835. **Kemmer - Lubowitz**
84   - **Price:** $707.19
85   - **Available from:** December 3, 2024
86   - **Available until:** May 16, 2025
87
886. **Jacobi, Wiegand, and Schamberger**
89   - **Price:** $755.09
90   - **Available from:** December 3, 2024
91   - **Available until:** October 19, 2025
92
93The most affordable option is the Gibson, Lueilwitz, and Batz hotel priced at $118.95. If you need further assistance or have preferences, feel free to let me know!
94✔ You: nope
95...
96Assistant: If you have any more questions or need further assistance in the future, feel free to reach out. Have a great trip to Miami!
97
98If you're all set, you may exit the session by typing 'exit'.
99✔ You: exit
100Goodbye!

The potential of this feature is HUGE! You can connect the model to virtually anything. Of course, it’s essential to keep security, performance, and user experience in mind. Additionally, there are opportunities to simplify access to this feature—perhaps through a library that abstracts much of the tool configuration and acts as a bridge between your custom API and the LLM you want to use.

Let me know your thoughts!