1 Usage

library(ellmer)
chat <- chat_ollama(model = "llama3.1:8b")

chat$chat("Tell me a statistics joke!")

Here's one:

Why did the statistician break up with his girlfriend?

Because he found her to be insignificant at the 0.05 level! (get it?)

1.1 Chat object

The created chat object can be used to continue the conversation.

chat$chat("Tell me another statistics joke!")

Here's another one:

A man walks into a library and asks the librarian, "Do you have any statistics 
on the number of people who read the average book?"

The librarian replies, "Let me look it up."

You can also use the chat object to get the chat history.

chat

<Chat Ollama/llama3.1:8b turns=4 tokens=82/80>
── user [16] ───────────────────────────────────────────────────────────────────
Tell me a statistics joke!
── assistant [35] ──────────────────────────────────────────────────────────────
Here's one:

Why did the statistician break up with his girlfriend?

Because he found her to be insignificant at the 0.05 level! (get it?)
── user [15] ───────────────────────────────────────────────────────────────────
Tell me another statistics joke!
── assistant [45] ──────────────────────────────────────────────────────────────
Here's another one:

A man walks into a library and asks the librarian, "Do you have any statistics on the number of people who read the average book?"

The librarian replies, "Let me look it up."

Or if you want to get it in a structured format, you can use the get_turns method.

chat$get_turns()

[[1]]
<Turn: user>
Tell me a statistics joke!

[[2]]
<Turn: assistant>
Here's one:

Why did the statistician break up with his girlfriend?

Because he found her to be insignificant at the 0.05 level! (get it?)

[[3]]
<Turn: user>
Tell me another statistics joke!

[[4]]
<Turn: assistant>
Here's another one:

A man walks into a library and asks the librarian, "Do you have any statistics on the number of people who read the average book?"

The librarian replies, "Let me look it up."

1.2 Interactive usage

You can use a chatbot via console:

live_console(chat)

Or alternatively, via the browser:

live_browser(chat)

2 Advanced usage

Note that these advanced features are not necessary supported by all LLMs.

2.1 Reading images

Suppose you have an image below.

The LLM can “read” the image like below.

chat <- chat_openai(model = "gpt-4o-mini")
chat$chat("Describe the image below.",
          content_image_url("https://emitanaka.org/workshop-text-analysis-2025/images/abs-logo.jpg"))

The image features a logo with the text "Australian Bureau of Statistics" 
prominently displayed. Above the text is a coat of arms, which includes an 
emblematic representation of a kangaroo and an emu on either side. The word 
"AUSTRALIA" is also included in a smaller font above the coat of arms. The 
overall design uses a simple color scheme with a white background and black 
text.

2.2 Reading PDF documents

Now next, suppose you have some PDF document. Some LLMs can read the content of the PDF document. This is currently not supported on the Open AI via ellmer but is supported on Gemini.

chat <- chat_google_gemini(model = "gemini-2.0-flash")
chat$chat("Summarise the PDF document below.",
          content_pdf_url("https://www.abs.gov.au/about/our-organisation/corporate-reporting/abs-annual-reports/2023-24%20ABS%20Annual%20Report_final.pdf"))

Here is a summary of the PDF document:

The PDF is the Annual Report for 2023-24 from the Australian Bureau of 
Statistics (ABS). It presents the ABS's achievements, audited Financial 
Statements, and Annual Performance Statement, focusing on:
*   Modernising data acquisition
*   Harnessing artificial intelligence
*   Towards the 2026 Census
*   Highlights key initiatives into 2024–25 and beyond.

The report highlights key areas including data modernisation, use of artificial
intelligence, the 2026 Census preparations, data governance, and management of 
human resources. The report details financial performance, strategic 
priorities, performance measures, and acknowledges contributions from staff, 
partners, and advisors. It complies with the Public Governance, Performance and
Accountability Act 2013 and is presented to the Assistant Minister for 
Competition, Charities and Treasury. It also features the organizational 
structure and other data relevant to the activities of the ABS.

2.3 Tool or function calling

A LLM is often restricted to the information it has been trained on, so may not be aware of the current context

chat <- chat_openai(model = "gpt-4o-mini")
chat$chat("How long ago was the Australian Federal election?")

The Australian Federal election was held on May 21, 2022. As of today, it has 
been approximately 1 year and 5 months since that election took place.

You can register a tool or function to be used by the LLM.

get_current_date <- function() as.character(Sys.Date())
chat$register_tool(tool(
  get_current_date,
  "Gets the current date."
))

And now when you ask the same question, the LLM can use the tool to get the answer.

chat$chat("How long ago was the Australian Federal election?")

◯ [tool call] get_current_date()

● #> 2025-06-03

The Australian Federal election was held on May 21, 2022. Today is June 3, 
2025. This means the election was held approximately 3 years and 15 days ago.

But notice that it doesn’t have the recent context so it’s still not right!

3 R Shiny apps

Shiny helps you to write web apps easily with R. This has proved to be massively useful for fast deployment of and proof of concept web apps for data science. You can integrate a chatbot interface using shinychat or use Shiny Assistant to help you write Shiny apps.

3.1 Chat assistant with `shinychat`

You can easily make a Shiny web app that incorporates a nice chat user interface using shinychat.

library(shiny)
library(shinychat)

ui <- fluidPage(
  chat_ui("chat")
)

server <- function(input, output, session) {
  chat <- ellmer::chat_openai(system_prompt = "You're a helpful statistics tutor.")
  
  observeEvent(input$chat_user_input, {
    stream <- chat$stream_async(input$chat_user_input)
    chat_append("chat", stream)
  })
}

shinyApp(ui, server)

You may notice that however the mathematics in LaTeX doesn’t render properly. We need to use a javascript library, like MathJax, to render the mathematics on the web, which can normally be injected using withMathJax(). However, due to an unwanted processing in the markdown text, this doesn’t work as intended (see discussion here). A hack to fix this is shown below.

library(shiny)
library(shinychat)
library(ellmer)
library(promises)
ui <- bslib::page_fillable(
  chat_ui("chat")
)

server <- function(input, output, session) {
  
  chat <- ellmer::chat_openai(system_prompt = "You're a helpful statistics tutor. Show maths with \\(...\\) for in-line math and $$...$$ for displayed with no spaces in between.")
  stream <- chat$stream_async("What is the mean?")
  stream2 <- coro::async_generator(function() {
    for (chunk in coro::await_each(stream)) {
      yield(gsub("\\", "\\\\", chunk, fixed = TRUE))
    }
  })()
  chat_append("chat", stream2) %...>% {
    insertUI("body", "beforeEnd", withMathJax(), immediate = TRUE)
  }
}

shinyApp(ui, server)

3.2 Shiny Assistant

Shiny Assistant is an AI assistant to help you build Shiny apps! See the blog post about it here and to use it, go to https://gallery.shinyapps.io/assistant.

🎉 Heh! You’ve reached the end of this exercise! You are flawless! Proceed to …

next slide

next exercise