.. Docker... It always was

Docker image is a WSL2...Linux port. Its not Windows, you're just using a third party app to run it in windows at the cost of resources to run the docker server. (which ive been doing for a long time, duh...) I mean its out on windows which makes development SIGNIFICANTLY easier. ;)@@geekazoid47

lol yeah I just saw the release right after this video posted. Good call. I currently have my GPU passed through to my windows VM for different compute things. Glad to see ollama on windows. I’ll definitely try this out.

@@geekazoid47 pain in the ass to setup for someone with no knowledge on setting up docker

@@3ace846 wdym docker is the easiest thing much better than regular commandline installing of things

What do you think about this model , what’s your experience?

Working on it - just having trouble getting a stable Ubuntu (with UI) running so i can fully test the end-to-end flow

@@TimCarambatTake your time, better to get it right then having to patch-it.

@@GregRutkowski I got an AppImage live on the main site now. Tested and built on Ubuntu 22, not sure how compatible that build system is with other distros, but I dont have inifinite linux machines (or time!) useanything.com/download

@@TimCarambat Time to test it out...

Ah, this is an error class i just refer to as "general knowledge creep". The only semi-good remedy is just a really strong system prompt explicitly detailing that only knowledge in the context should be used and to not use it's existing understanding of a game on how rules for this game should work - or something along those lines. System prompt can def fix this and depending on the model will vary in performance.

Unlikely because it cant fit into our stack (since its Python). The idea of the framework however is not impossible to support

Already live in main repo. Will be live in next weekly release

If on windows you can run LMStudio (just like ollama but with a UI) and LocalAI (requires docker to run). Outside of that you can also turn to hosted LLMs on OpenAI, Azure OpenAI, Anthropic, Google Gemini Together AI, and Mistral

Being able to fine tune off the chat and such is on the roadmap, but is still conceptual since 99% of people cannot train a model on their device. So we may integrate or connected to some provider which can enable this to happen more easily. We allow you to export the chats in a fine-tune format already to encourage this behavior and id love to close the loop on it

@@TimCarambat Thanks Tim appreciate you.

You know it's funny, we allow pulling in and embedding of whole github repos, but not the independent code files. I will work on resolving that for you - thanks for pointing it out 😄

@@TimCarambat you are absolutely welcome, if i helped even 0.01% it made my day!

Reply just to say we support this now (and have for a few weeks now!)

You can scrape websites with the built in scraper but its not "internet connected". Not like an agent-LLM. CodeLlama for local LLM. Claude-3 Opus for closed source

Same. Whats the issue? Have you found any solutions?

@@Musaddique_MMALI I reinstalled today because I formatted my computer. Now is working fine.

You absolutely could but youll want to run our Docker offering because it is meant for that. The desktop app is really a local "single-player" solution. The docker app has a full API, multi-user support, etc etc

Been working on it this week! Looks like the app image should be good by next release, likely next week

Chat is general knowledge + documents Query is just documents. In this case, since each query had used context - the results would have been the same

Not currently, it would be nice to add though for obvious reasons

How can I get a list 📝 of models that work on OLLAMA?

@@ajarivas72 Lllama3 now is not bad, but still solar and Mixtral are good too.

Where is the disconnect for you? This video is quite old now and we have improved the product a lot since

The one in the uploader is currently just the url. Understandably that is not great for large url ingestion and there is a piece of work being done to add deep scraping (or at least multi link, possibly agent based) to the tool.

Actually I reported a bug, maybe it's not a bug but something I set wrong. Sorry for false reporting then. github.com/Mintplex-Labs/anything-llm/issues/821, thanks again for making this great app.

​@@Gabriecielowhat kind of use case you have tested with the embed files (what format?)? I am interested in generating a document based on data within multiple local files. So you think it's feasible with ollama/AnythingLLM?

@@m.bouanane4455 I tried PDF, more text books. Mitral 7B and Gemma tested, Gemma lost a lot to Mitral. And it works with multiple docs too. One thing I felt is AI needs my questions to drive him read and think deeper into the book.

Tested with PDF mainly, works fine. And we could easily switch all open-source LLMs. For using AnythingLLM to generate doc, you should ask Tim, I only tried use interface, didn't know if API is available or other tools for you to drive. @@m.bouanane4455

The docker image comes with a full REST API. It is accessed via TCP/IP though. You can wrap it though on the Android side with Retrofit or something like that

Just downloading the page. No agent support currently

The docker version has multi-user with auth :)

The thread feature if available in the latest version of the desktop app - so you can redownload and just overwrite the application and youll still keep your data. It looks like you have plenty of power and memory. "Failed to fetch" with LMStudio almost always means that the endpoint being used or the API key (if applicable) is incorrect. The LMStudio endpoint should be somthing like `localhost:1234/v1` in AnythingLLM. Make sure you started the LMStudio inference server in that application as well since it does not start on boot. medium.com/@ingridwickstevens/running-a-local-openai-compatible-mixtral-server-with-lm-studio-cd602efbf808

​@@TimCarambatI don't think it's LM Studio related at all, cause I used it with multiple tools without trouble, as I said i used localhost:1234/v1 (you can't set the API key in AnythingLLM), but chat work fine in AnythingLLM, but as soon I try to chat or query after embedding anything I get that error (Could not respond to message. An error occurred while streaming response. Failed to fetch.) Update: I installed the latest. First tried without deleting anythingllm preference folder and got a fail in trying to save and embed the document suggest trying a different schema than LanceDB. Since the error was different I thought maybe the preferences have compatibility issues, so I cleaned up the preference folder and started from scratch. And got the original message on the arxiv pdf (but no failure on embedding and saving this time), this partially froze the process as I explained above, I also got multiple wmic.exe and conhost.exe processes, a few times multiple powershell.exe spawning on my system and high CPU usage from process: WmiPrvSE.exe. So I restarted and tried uploading a new story from the web from a basic webpage (longstoryshort.squarespace_com/great-example) and got an error when attempting to move the doc to the workspace(Save and embed): It fails again and suggest trying a different schema than LanceDB but at least it doesn't freeze (it still spawns multiple processes). So I think it's a problem with the LanceDB wheter it does fail to save and embed or not. I will try installing Chroma and see what happens, will try Ollama as well, I would still check.

We have a docker image if that works for you??

Thank you Tim I'll look into that!

AppImage is live for AnythingLLM useanything.com/download And Ollama supports a linux download now as well. I think its just an install script

😍

@@TimCarambat:D

That is built into anythingllm automatically. Outside of that you have to manage it directly in ollama

@@TimCarambat "outside of that" what do you mean by that?

@@ogr3dblade6 I mean outside of just using anythingllm, sorry

Literally 5 minutes later, it stops calling me by my name and it forgets who i am.

@@TimCarambat is the session RAG feature flawed or do you think i messed something up?

Yes, absolutely

Does docker not satisfy that need? Working on an AppImage but the docker image is so much more featured than the desktop client

Are you on an intel-based processor with no GPU? If so this is par for the course and normal.Bigger model can lead to even longer time-to-first-token or crashes

Go to "Vector database" and select LanceDB. Not sure how you got to that state since lancedb is the default, but that would fix it.

Thank you very much. I really appreciate it. I will try it out tonight. Have a great day. Best regards

Yes, it's the same repo as the main app, it's just the desktop branch. github.com/Mintplex-Labs/anything-llm

@@TimCarambat Thank you.

Do they have an api for that tool?

Ollama in docker does for sure work, some do it so its def a configuration issue between the containers and your host relaying the connection

Anythingllm is a chat UI and tool suit for multiple types of LLMs! Gpt-4, mistral, anthropics Claude, Google gemini, and local llms! Ollama is a tool that helps you download and run local LLMs but anything LLM can connect to it and give you a nice UI for chatting with your documents and more. AnythingLLM isn't an LLM, it's a tool for LLMs. I didn't make that clear in the video, but ollama is just a simple way to run local LLMs on your desktop/laptop

@@TimCarambatdo you have a video showing how to just use anything LLM and your preferred tool, ollama or LM studio on just one computer? For example, if I just have 1 windows computer how to do everything from start to finish using your tool and whichever one of these tools you prefer the best? So in other words, install, how to get the LLM, etc I know you have that one video but it involved using a server. And which do you prefer LM studio or Ollama?

And we support those models for embedding now as well!

AppImage should be live next week

AppImage is live useanything.com/download

We have a docker image!

Where?​@@TimCarambat

@@theratexperiment hub.docker.com/r/mintplexlabs/anythingllm github.com/Mintplex-Labs/anything-llm/blob/master/docker/HOW_TO_USE_DOCKER.md

This can be resolved by settings you can configure. The default settings work for most things, but we allow you to modify them because sometimes given your model, dataset, embedder selection, or more - the default configs may not work. Common fixes to the LLM still missing the point when querying and documents are embedded: - Change the Workspace Vector database "Document similarity" to "No Restriction" - Pin the document to the workspace, which ensures full-text comprehension, but obviously consumes many more tokens and takes much longer to yield a response.