Ollama - (in)Secure IT

Sign in Subscribe

Ollama

A collection of 6 posts

Running Local LLMs on Older Hardware?

Running Local LLMs on Older Hardware?

As part of this local LLM series, I wanted to know the lower limits of being able to self-host and potentially share demos or development frameworks with my team. Given the acceptable baseline discussion in my previous post, we'll use that to audition the hardware we come across.

Summary: GPT-4o vs Local Language Models Evaluation Results

Summary: GPT-4o vs Local Language Models Evaluation Results

This entry was intended to be Part-2 of my most recent report but realized on review that it is probably the natural follow-up to the other writings on the current state of LLMs (we've not even gone down the road of Small Language Models, yet!) in the first

Measuring Azure OpenAI GPT-4o vs Local LLM Models

GPT-4o Featured

Measuring Azure OpenAI GPT-4o vs Local LLM Models

Microsoft Build 2024 just concluded last week and one of the highlights was the catch-up release of OpenAI GPT-4o alongside a new hardware line-up of Copilot+ PCs running Windows on ARM with Qualcomm Snapdragon X chips under the hood. I'll address the huge x86 elephants that were in

Extending Llama-3 8B with 1-million context by Gradient

Extending Llama-3 8B with 1-million context by Gradient

Wow, things are heating up on this latest Meta model. I was looking to turn things up a notch on my local LLM workflow which I finally got working and gotten usable responses using the llama3:latest model from Ollama. For some reason, llama3:text was giving me some loopy

Meta Llama 3 vs Ollama Performance

Meta Llama 3 vs Ollama Performance

In the middle of me taking on the challenge of a small full-stack project by my lonesome by rapidly developing a parallel alternative to one in production backed by more than a dozen engineers, I started looking to get a bit more embedding performance or alternatives to the "text-embedding-3-large&

Ollama Performance Comparison Chart

Ollama Performance Comparison Chart

I stumbled upon Ollama while looking for ways to run large language models (LLMs) locally for research at work some time last year during the initial explosion of the interest in ChatGPT. Being a long time Linux dabbler all my GPU have almost always been from Team Red (AMD) unless