library(ellmer)
While not all LLMs are the same, there are some parameters that are common across many of the LLM. Note that the names of these parameters (and its exact interpretation and specification) may differ across LLM. Ultimately, you have to dig into the documentation (or code) to find what parameters are available to you and how to use it.
I’ll be using Ollama with llama3.1:8b for the examples below, but you can replace these chat_ollama() with your preferred vendor and model instead.
We’ll look at the use of different model parameters. For each parameter, let’s consider some “actors” and see what the response is like.
1 temperature
A higher temperature results in a diverse response, while a lower temperature results in selection of more probable texts. In general, select a lower temperature for more predictable and sensible responses.
be_statistician <- "Respond like a statistician but keep it concise. No more than 80 words."
sensible_statistician <- chat_ollama(system_prompt = be_statistician,
seed = 1,
model = "llama3.1:8b",
api_args = list(temperature = 0))
creative_statistician <- chat_ollama(system_prompt = be_statistician,
seed = 1,
model = "llama3.1:8b",
api_args = list(temperature = 40))
sensible_statistician$chat("My p-value is 0.049. Is this significant?")A p-value of 0.049 is indeed statistically significant, typically considered so
at the 5% significance level (α = 0.05). However, it's worth noting that some
researchers consider a more stringent threshold, such as α = 0.01 or Bonferroni
correction, to account for multiple comparisons.creative_statistician$chat("My p-value is 0.049. Is this significant?") Nesting ambiguity concerns - not strictly by intention rather your α-risk
remains unfailily protected still around significance value (concordally of) α
set on convention <p. The borderline effect is however suggestive enough not a
clear conclusion to disprove 'Null'.
Alpha, P > a probability for you as your outcome depends: reject if no p=. Not
an affirmation by statistics however in scientific results the choice might
indeed reflect...a careful interpretation's necessity rather! Further steps to
interpret cautiously still remains! .2 top_p
top_p should be a value between 0 and 1 (inclusive). A value of less than 1 ensures that the most unlikely responses will not get selected, however high the temperature is. You can use top_p combined with a higher temperature to get restraint creative responses.
restraint_statistician <- chat_ollama(system_prompt = be_statistician,
seed = 1,
model = "llama3.1:8b",
api_args = list(temperature = 40,
top_p = 0.5))
restraint_statistician$chat("My p-value is 0.049. Is this significant?") A value of 0.049 is just barely below the conventional significance threshold
of 0.05. However, due to the "p-hacking" issue, some researchers consider
values between 0.04 and 0.06 as not fully reliable. It's a borderline case. I'd
recommend conducting a power analysis or exploring alternative explanations
before concluding significance.3 seed
The seed ensures that the same random sample is chosen.
be_pirate <- "Respond like a pirate but keep it to 10 words."
pirate1 <- chat_ollama(model = "llama3.1:8b",
system_prompt = be_pirate,
seed = 1)
pirate2 <- chat_ollama(model = "llama3.1:8b",
system_prompt = be_pirate,
seed = 2)
pirate3 <- chat_ollama(model = "llama3.1:8b",
system_prompt = be_pirate,
seed = 1)Above pirate1 and pirate3 have the same seed, whilst pirate2 have a different seed. The other model parameters are all the same (default) for all three pirates.
pirate1$chat("Hi!")Matey, lovely day fer sailin', be ye?pirate2$chat("Hi!")Arrr, welcome aboard me ship, matey, come aboard!pirate3$chat("Hi!")Matey, lovely day fer sailin', be ye?4 stop
stop words specify when the LLM should stop generating new texts.
pirate4 <- chat_ollama(model = "llama3.1:8b",
system_prompt = be_pirate,
seed = 1,
api_args = list(stop = "Arrr"))Notice below, the sentence stops midway as the next word was likely “Arrr”, the stop word.
pirate4$chat("Hi")Me hearty, 5 max_tokens or max_completion_tokens
Open AI originally used max_tokens but deprecated this so now it is named max_completion_tokens. max_tokens still works for older models but you may need to use max_completion_tokens for future models. Ollama uses max_tokens.
haiku <- chat_ollama(model = "llama3.1:8b",
seed = 1,
system_prompt = "Be poetic.",
api_args = list(max_tokens = 17))
short <- chat_ollama(model = "llama3.1:8b",
seed = 1,
system_prompt = "Be poetic.",
api_args = list(max_tokens = 5))
haiku$chat("Tell me more about p-values?")Oh, the humbling tale of p-values! A statistical measure so oft cited,short$chat("Tell me more about p-values?")Oh, the humbling🎉 Yee-haw! You’ve reached the end of this exercise! You are awe-inspiring! Proceed to …