Tag: LLMs

Categories
Life Style

AI & robotics briefing: LLMs harbour hidden racism

[ad_1]

Hello Nature readers, would you like to get this Briefing in your inbox free every week? Sign up here.

A laptop user is typing on the Google Bard AI chatbot webpage.

Some models are more likely to associate African American English with negative traits than Standard American English.Credit: Jaap Arriens/NurPhoto via Getty

Some large language models (LLMs), including those that power chatbots such as ChatGPT, are more likely to suggest the death penalty to a fictional defendant presenting a statement written in African American English (AAE) compared with one written in Standardized American English. AAE is a dialect spoken by millions of people in the United States that is associated with the descendants of enslaved African Americans. “Even though human feedback seems to be able to effectively steer the model away from overt stereotypes, the fact that the base model was trained on Internet data that includes highly racist text means that models will continue to exhibit such patterns,” says computer scientist Nikhil Garg.

Nature | 5 min read

Reference: arXiv preprint (not peer-reviewed)

A drug against idiopathic pulmonary fibrosis, created from scratch by AI systems, has entered clinical trials. Researchers at Insilico Medicine identified a target enzyme using an AI system trained on patients’ biomolecular data and scientific literature text. They then used a different algorithm to suggest a molecule that would block this enzyme. After some tweaks and laboratory tests, researchers had a drug that appeared to reduce inflammation and lung scarring. Medicinal chemist Timothy Cernak says he was initially cautious about the results because there’s a lot of hype about AI-powered drug discovery. “I think Insilico’s been involved in hyping that, but I think they built something really robust here.”

Chemical & Engineering News | 4 min read

Reference: Nature Biotechnology paper

Researchers built a pleurocystitid robot to investigate how the ancient sea creature moved. Pleurocystitids lived 450 million years ago and were probably one of the first echinoderms (animals including starfish and sea urchins) that could move from place to place using a muscular ‘tail’. The robot moved more effectively on a sandy ‘seabed’ surface when it had a longer tail, which matches fossil evidence that pleurocystitids evolved longer tails over time.

Ars Technica | 5 min read

Reference: PNAS paper

Image of Rhombot robot testbed inspired by anatomy of pleurocystitid.

The tail of the pleurocystitid replica (nicknamed ‘Rhombot’) was built out of wires that contract in response to electrical stimulation to simulate the flexibility and rigidity of a natural muscular tail.(Carnegie Mellon University – College of Engineering)

Features & opinion

Scientists hope that getting AI systems to comb through heaps of raw biomolecular data could reveal the answer to one of the biggest biological questions: what does it mean to be alive? AI models could, with enough data and computing power, build mathematical representations of cells that could be used to run virtual experiments — as well as map out what combination of biochemistry is required to sustain life. Researchers could even use it to design entirely new cells, that, for example, can explore a diseased organ and report on its condition. “It’s very ‘Fantastic Voyage’-ish,” admits biophysicist Stephen Quake. “But who knows what the future is going to hold?”

The New York Times | 9 min read

The editors of Nature Reviews Physics and Nature Human Behaviour have teamed up to explore the pros and cons of using AI systems such as ChatGPT in science communication. Apart from making up convincing inaccuracies, write the editors, chatbots have “an obvious, yet underappreciated” downside: they have nothing to say. Ask an AI system to write an essay or an opinion piece and you’ll get “clichéd nothingness”.

In Nature Human Behaviour, six experts discuss how AI systems can help communicators to make jargon understandable or translate science into various languages. At the same time, AI “threatens to erase diverse interpretations of scientific work” by overrepresenting the perspectives of those who have shaped research for centuries, write anthropologist Lisa Messeri and psychologist M. J. Crockett.

In Nature Reviews Physics, seven other experts delve into the key role of science communication in building trust between scientists and the public. “Regular, long-term dialogical interaction, preferably face-to-face, is one of the most effective ways to build a relationship based on trust,” notes science-communication researcher Kanta Dihal. “This is a situation in which technological interventions may do more harm than good.”

Nature Reviews Physics editorial | 4 min read, Nature Human Behaviour feature | 10 min read & Nature Reviews Physics viewpoint | 16 min read

Technology journalist James O’Malley used freedom-of-information requests to unveil how one of London’s Underground stations spent a year as a testing ground for AI-powered surveillance. Initially, the technology was meant to reduce the number of people jumping the ticket barriers, but it was also used to alert staff if someone had fallen over or was spending a long time standing close to the platform edge. Making every station ‘smart’ would undoubtedly make travelling safer and smoother, argues O’Malley. At the same time, there are concerning possibilities for bias and discrimination. “It would be trivial from a software perspective to train the cameras to identify, say, Israeli or Palestinian flags — or any other symbol you don’t like.”

Odds and Ends of History blog | 14 min read

Image of the week

An animated sequence of the Jellyfish biohybrid robot swimming with an attached hemi-ellipsoid forebody.

Simon R Anuszczyk and John O Dabiri/Bioinspir. Biomim. (CC BY 4.0)

A 3D-printed ‘hat’ allows this cyborg jellyfish to swim almost five times faster than its hat-less counterparts. The prosthesis could also house ocean monitoring equipment such as salinity, temperature and oxygen sensors. Scientists use electronic implants to control the animal’s speed and eventually want to make it fully steerable, in order to gather deep ocean data that can otherwise only be obtained at great cost. “Since [jellyfish] don’t have a brain or the ability to sense pain, we’ve been able to collaborate with bioethicists to develop this biohybrid robotic application in a way that’s ethically principled,” says engineer and study co-author John Dabiri. (Popular Science | 3 min read)

Reference: Bioinspiration & Biomimetics paper

Quote of the day

Machine-learning engineer Rick Battle says that chatbots’ finicky and unpredictable performance depending on how they’re prompted makes sense when thinking of them as algorithmic models rather than anthropomorphized entities. (IEEE Spectrum | 12 min read)

[ad_2]

Source Article Link

Categories
News

How to fine tune large language models (LLMs) with memories

How to fine tune LLMs with memories

If you would like to learn more about how to fine tune AI language models (LLMs) to improve their ability to memorize and recall information from a specific dataset. You might be interested to know that the AI fine tuning process involves creating a synthetic question and answer dataset from the original content, which is then used to train the model.

This approach is designed to overcome the limitations of language models that typically struggle with memorization due to the way they are trained on large, diverse datasets. To explain the process in more detail Trelis Research has created an interesting guide and overview on how you can find tune large language models for memorization.

Imagine you’re working with a language model, a type of artificial intelligence that processes and generates human-like text. You want it to remember and recall information better, right? Well, there’s a way to make that happen, and it’s called fine-tuning. This method tweaks the model to make it more efficient at holding onto details, which is especially useful for tasks that need precision.

Language models are smart, but they have a hard time keeping track of specific information. This problem, known as the “reversal curse,” happens because these models are trained on huge amounts of varied data, which can overwhelm their memory. To fix this, you need to teach the model to focus on what’s important.

Giving LLMs memory by fine tuning

One effective way to do this is by creating a custom dataset that’s designed to improve memory. You can take a document and turn it into a set of questions and answers. When you train your model with this kind of data, it gets better at remembering because it’s practicing with information that’s relevant to what you need.

Now, fine-tuning isn’t just about the data; it’s also about adjusting certain settings, known as hyperparameters. These include things like how much data the model sees at once (batch size), how quickly it learns (learning rate), and how many times it goes through the training data (epoch count). Tweaking these settings can make a big difference in how well your model remembers.

Here are some other articles you may find of interest on the subject of large language models and fine-tuning :

Fine tuning large language models

Choosing the right model to fine-tune is another crucial step. You want to start with a model that’s already performing well before you make any changes. This way, you’re more likely to see improvements after fine-tuning. For fine-tuning to work smoothly, you need some serious computing power. That’s where a Graphics Processing Unit (GPU) comes in. These devices are made for handling the intense calculations that come with training language models, so they’re perfect for the job.

Once you’ve fine-tuned your model, you need to check how well it’s doing. You do this by comparing its performance before and after you made the changes. This tells you whether your fine-tuning was successful and helps you understand what worked and what didn’t. Fine-tuning is a bit of an experiment. You’ll need to play around with different hyperparameters and try out various models to see what combination gives you the best results. It’s a process of trial and error, but it’s worth it when you find the right setup.

To really know if your fine-tuned model is up to par, you should compare it to some of the top models out there, like GPT-3.5 or GPT-4. This benchmarking shows you how your model stacks up and where it might need some more work.

So, if you’re looking to enhance a language model’s memory for your specific needs, fine-tuning is the way to go. With a specialized dataset, the right hyperparameter adjustments, a suitable model, and the power of a GPU, you can significantly improve your model’s ability to remember and recall information. And by evaluating its performance and benchmarking it against the best, you’ll be able to ensure that your language model is as sharp as it can be.

Filed Under: Guides, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.

Categories
News

ChatHub AI lets you run large language models (LLMs) side-by-side

ChatHub lets you access AI models side-by-side

If you are searching for a way to run AI models in the form of large language models (AI) side-by-side to see which provides the best results. You might be interested in a new application called ChatHub that allows you to talk to artificial intelligence (AI) as easily as chatting with a friend.  At the heart of ChatHub is its ability to connect you to several LLMs all in one place. Use ChatGPT, Bing Chat, Google Bard, Claude 2, Perplexity, and other open-source large language models as you need.

This means you don’t have to jump from one website to another to try out different AI models. You can see how up to six LLMs perform right next to each other, comparing their creativity, speed, and accuracy. This not only saves you time but also helps you get the best results by combining the strengths of each model.

ChatHub has been specifically designed to incorporate features that make your life easier when using AI, like the ability to quickly copy information, track your history, and search swiftly through past interactions. These aren’t just convenient; they give you more control over how you use AI, making your work more efficient. The development team responsible for creating the platform of also created a Chrome extension.

Using ChatHub to access different AI models

One of the coolest things about ChatHub is its prompt library. It’s full of prompts created by the community and a tool that helps you come up with your own. This is a huge help, whether you’re new to AI or you’ve been using it for a while. It guides you in asking the right questions to get the most useful answers from the AI.

Here are some other articles you may find of interest on the subject of AI models :

Easily switch between AI models

 

ChatHub is all about giving you choices. You can switch between popular LLMs depending on what you need at the moment. This flexibility means that the platform can adapt to a wide range of tasks, whether you’re writing a report, analyzing data, or just exploring what AI can do. For those who need even more customization, ChatHub has an API integration feature. This lets you add your own chat models using API keys. It opens up a world of possibilities for tasks that are specific to your needs or your business.

Some LLMs on ChatHub have special skills, like recognizing images or browsing the web. These abilities take what you can do with AI to a whole new level. You could analyze pictures or pull information from the internet, making ChatHub a versatile tool in your AI arsenal.

Now, it’s true that ChatHub might not have every single feature that some of its competitors offer. For example, OpenAI’s ChatGPT Plus has some functionalities that you won’t find on ChatHub. But what sets ChatHub apart is its pricing. You pay once to get a license, and you don’t have to worry about monthly subscriptions. Plus, they sometimes have discounts, which can make it a more affordable option.

So, if you’re looking to dive into the world of AI, or if you’re already swimming in it and need a better tool, ChatHub could be just what you need. It’s designed to make working with AI simpler and more effective, whether you’re using it for business, research, or personal projects. With its user-friendly interface and a wide range of features, ChatHub is ready to take your AI experience to the next level.

Filed Under: Technology News, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.

Categories
News

Ollama for Windows now available to run LLM’s locally

Ollama for Windows now available to run LLM's locally

Microsoft Windows users who have been patiently waiting to use the fantastic Ollama app that allows you to run large language models (LLMs) on your local machine. Will be pleased to know that the Ollama development team has now released a Windows version. Previously only available on macOS and Linux, Ollama is now available to run on PCs running Windows 10 and above.

Imagine a tool that transforms your Windows 10 computer into a powerhouse of artificial intelligence capabilities. This is Ollama, and it brings the sophisticated world of large language models (LLMs) straight to your desktop. With its release for Windows 10, users can now tap into the same advanced AI features that have been enhancing productivity on other platforms.

The standout feature of Ollama is its GPU acceleration. By utilizing NVIDIA GPUs,, the application can process complex language and vision models at breakneck speeds. This acceleration means that tasks which would typically take longer to compute are now completed in a fraction of the time, allowing you to work more efficiently and effectively.

Ollama now available on Windows

But speed isn’t the only advantage Ollama offers. It comes with a comprehensive library of models that cater to a variety of needs. Whether you’re working with text or images, Ollama has a model that can help. These models are not only powerful but also easy to integrate into your existing workflow. The application’s drag-and-drop functionality makes it simple to use, and its always-on API ensures that it connects seamlessly with the other tools you rely on.

Here are some other articles you may find of interest on the subject of Ollama

Privacy and control are paramount in today’s digital landscape, and Ollama addresses these concerns head-on. The application is compatible with OpenAI-based tools, allowing you to run local models and keep your data secure. This means that you can enjoy the benefits of AI without compromising your privacy or losing control over your information.

Setting up Ollama is straightforward. The installation process is guided by the OllamaSetup.exe installer, which means you can start exploring AI-powered features without any hassle. The developers are committed to improving the application, and regular updates are released to ensure that Ollama continues to meet the evolving needs of its users.

The creators of Ollama understand the importance of community feedback. They encourage users to share their experiences and suggestions, which helps shape the future development of the application. For those seeking support or wishing to connect with like-minded individuals, there’s a dedicated Discord channel where users can engage with each other and the development team.

Ollama for Windows 10 is more than just an application; it’s a comprehensive platform that simplifies the integration of AI into your daily tasks. With features like GPU acceleration, a vast model library, and seamless integration with OpenAI tools, Ollama is tailored to enhance your productivity and expand your capabilities. Its user-friendly design, coupled with a commitment to user-driven development, positions Ollama as a vital tool for anyone interested in leveraging the power of AI on their Windows system.

Filed Under: Technology News, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.

Categories
News

How to use ChatGPT and LLMs for data extraction

using ChatGPT and large language models for data extraction

Artificial intelligence (AI) has taken huge leaps forward in the last 18 months with the development of sophisticated large language models. These models, including GPT-3.5, GPT-4, and open source LLM OpenChat 3.5 7B, are reshaping the landscape of data extraction. This process, which involves pulling out key pieces of information like names and organizations from text, is crucial for a variety of analytical tasks. As we explore the capabilities of these AI tools, we find that they differ in how well they perform, how cost-effective they are, and how efficiently they handle structured data formats such as JSON and YAML.

These advanced models are designed to understand and process large volumes of text in a way that resembles human cognition. By simply entering a prompt, they can filter through the text and deliver structured data. This makes the task of extracting names and organizations much smoother and allows for easy integration into further data analysis processes.

Data Extraction using ChatGPT and OpenChat locally

The examples below show how to save your extracted data to JSON and YAML files. Because they are easy to read and work well with many programming languages. JSON is particularly good for organizing hierarchical data with its system of key-value pairs, while YAML is preferred for its straightforward handling of complex configurations.

Here are some other articles you may find of interest on the subject of using large language models for data extraction and analysis :

However, extracting data is not without challenges. Issues like incorrect syntax, unnecessary context, and redundant data can affect the accuracy of the information retrieved. It’s crucial to adjust these large language models carefully to avoid these problems and ensure the responses are syntactically correct.

When we look at different models, proprietary ones like GPT-3.5 and GPT-4 from OpenAI are notable. GPT-4 is the more advanced of the two, with better context understanding and more detailed outputs. OpenChat 3.5 7B offers an open-source option that is less expensive, though it may not be as powerful as its proprietary counterparts.

Data extraction efficiency can be greatly improved by using parallel processing. This method sends multiple extraction requests to the model at the same time. It not only makes the process more efficient but also reduces the time needed for large data extraction projects.

Token Costs

The cost of using these models is an important factor to consider. Proprietary models have fees based on usage, which can add up in big projects. Open-source models can lower these costs but might require more setup and maintenance. The amount of context given to the model also affects its performance. Models like GPT-4 can handle more context, which leads to more accurate extractions in complex situations. However, this can also mean longer processing times and higher costs.

Creating effective prompts and designing a good schema are key to guiding the model’s responses. A well-crafted prompt can direct the model’s focus to the relevant parts of the text, and a schema can organize the data in a specific way. This is important for reducing redundancy and keeping the syntax precise.

Large language models are powerful tools for data extraction, capable of quickly processing text to find important information. Choosing between models like GPT-3.5, GPT-4, and OpenChat 3.5 7B depends on your specific needs, budget, and the complexity of the task. With the right setup and a deep understanding of their capabilities, these models can provide efficient and cost-effective solutions for extracting names and organizations from text.

Filed Under: Guides, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.

Categories
News

AutoGen Studio running purely on local LLMs

AutoGen Studio running purely on local LLMs

If you are interested in running artificial intelligence and AI models locally the ability  to integrate local large language models (LLMs) into your own systems for personal or business use. AutoGen Studio, a cutting-edge AI platform, has made this possible, allowing users to harness the power of LLMs directly within their workspace. This integration is a significant step forward for those who wish to maintain control over their data while benefiting from the advanced capabilities of language models.

AutoGen Studio has introduced a new feature that allows users to replace the default GPT-4 model with an open-source alternative. This gives users the freedom to customize their AI tools and retain data sovereignty, a critical concern for many businesses and individuals who are wary of storing sensitive information on external servers.

“With AutoGen Studio, users can rapidly create, manage, and interact with agents that can learn, adapt, and collaborate. As we release this interface into the open-source community, our ambition is not only to enhance productivity but to inspire a level of personalized interaction between humans and agents”  explains the Microsoft team over on the official GitHub blog.

To begin using this feature, users must first download and install LM Studio, a versatile platform that supports various operating systems including macOS, Windows, and Linux. The installation process is straightforward, with a user-friendly guide to help get LM Studio up and running on your device.

AutoGen Studio running local large language models (LLMs)

Once installed, the next step is to set up a local server. This server will act as the central hub for your chosen LLM, providing an API endpoint that connects AutoGen Studio with the language model. This connection is vital for the seamless operation of the AI tools within your workspace. LM Studio offers a selection of LLMs to choose from, each with its own strengths and suited for different project requirements.

For example, the Hermes 2.5 mral 7B model is a versatile option that can be downloaded and used as the driving force behind your linguistic tasks. Once again thanks to Prompt Engineering  for creating a fantastic overview and demonstration of how AutoGen Studio can be run purely on local large language models opening up a wide variety of possibilities and applications for both personal and business use.

Here are some other articles you may find of interest on the subject of large language models and AutoGen :

After selecting and setting up your LLM, you’ll need to configure AutoGen Studio. This involves creating new agents and workflows that will utilize the capabilities of your local LLM. These agents and workflows are at the heart of AutoGen Studio’s functionality, enabling users to automate a wide range of tasks with the intelligence of the LLM.

Before deploying your agents, it’s wise to test them in AutoGen Studio’s playground. This simulated environment allows you to refine your workflows and ensure that your agents perform as expected. It’s an essential step in the development process, helping to iron out any issues before going live.

It’s important to be aware of the limitations that come with open-source LLMs. Some may not have the capability to generate visuals or perform function calls. Understanding these limitations is key to successfully integrating LLMs into your projects. For tasks that require these advanced features, you may need to look into more sophisticated open-source LLMs.

For those with projects that demand more complex functionalities, the open-source LLM ecosystem offers a range of models that may fit the bill. Exploring this ecosystem can lead to the discovery of a model that is capable of handling the intricate tasks required by your project.

The integration of local LLMs with AutoGen Studio through LM Studio provides users with powerful language modeling tools that can be customized to meet specific needs while maintaining privacy and control over data. By following the steps outlined above, users can create a tailored AI solution that aligns with their unique requirements. This integration is a testament to the flexibility and adaptability of AI technology, offering a new level of customization for those looking to incorporate AI into their workflows.

Filed Under: Guides, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.

Categories
News

Open Interpreter update lets large language models LLMs run code

Open Interpreter lets large language models (LLMs) run code

The software development community has just witnessed the arrival of Open Interpreter 0.2.0, an updated tool that promises to make coding and task management much more efficient. Open Interpreter lets large language models (LLMs) run code (Python, Javascript, Shell, and more) locally. You can chat with Open Interpreter through a ChatGPT-like interface in your terminal

This new version is packed with features that are designed to help programmers work smarter and faster. One of the most exciting additions is a natural language interface, which lets you give commands to your computer in plain English. This means that even those who are new to coding can start performing complex tasks right away, and experienced coders can work more quickly than before.

“The New Computer Update is the most significant upgrade to Open Interpreter since 0.1.0. Almost every component has been rewritten to support our project’s objective—building a standard interface between language models and computers.”

Open Interpreter update

  • Introduction of the Computer API by Open Interpreter: Open Interpreter developed a real-time code execution environment for language models in 2023. They introduced an API allowing language models to control basic computer functions like display, mouse, and keyboard. This includes taking screenshots, clicking and moving the mouse based on on-screen text or icons, and accessing clipboard contents.
  • OS Mode Feature: Open Interpreter enables a feature where users can command their computer graphically using the Computer API. This is done through a simple command (interpreter --os) and is compatible with various multimodal language models, including local vision models.
  • LMC Messages for Enhanced Communication: Open Interpreter has upgraded its messaging format to support the new Language Model Computer architecture. This new format includes additional information and introduces a ‘computer’ role, facilitating enhanced communication between the assistant and the computer, such as executing code, processing images, and sending confirmation messages.
  • Computer Module Independence: The Computer module is now independent of Open Interpreter’s core. This allows users to run code independently in the same Python environment used by the interpreter. Users can define functions, variables, log into services, and have control over the computer’s programming languages, enhancing flexibility and customization.

A key feature of Open Interpreter 0.2.0 is the OS mode, which automates repetitive tasks. This is a big time-saver for developers, who can now automate the mundane parts of their work, like typing out the same commands over and over or moving files around. This leaves them free to concentrate on the more creative and complex aspects of their projects. The real-time code execution environment is another highlight, providing instant feedback on how well your code is working. This is crucial for finding and fixing errors quickly, which is a big part of a developer’s job.

Open Interpreter new features

The new version of Open Interpreter also supports multiple programming languages. Whether you’re working in Python, JavaScript, or shell scripting, this tool has you covered. This is great for developers because it means they can choose the best language for each task, without having to switch between different tools. The updated graphical user interface (GUI) is also worth mentioning. It’s been redesigned to be more intuitive, which makes it easier for developers to find their way around the software and use all of its features.

Here are some other articles you may find of interest on the subject of large language models :

One of the more technical updates in Open Interpreter 0.2.0 is the integration of a Computer API. This allows the software to interact directly with the computer’s operating system, which can lead to more advanced and responsive applications. It’s a big step forward for developers who want to push the boundaries of what’s possible with their software. The new LMC messaging format is another important addition. It standardizes the way language models and computers talk to each other, which should cut down on mistakes and make the whole process more efficient.

The modular architecture of Open Interpreter 0.2.0 is also worth noting. It means that developers can run different parts of the software independently, which gives them the flexibility to set up their development environment exactly how they like it. This can make coding a much more pleasant experience. Lastly, the platform now lets you define functions and variables, and even log into services. This makes it a more complete environment for running code, and the addition of custom language support means it can be used for a wider range of projects.

Overall, Open Interpreter 0.2.0 is a sophisticated tool that’s designed to make life easier for developers. With its new features and improvements, it’s a powerful asset for anyone who writes code, and it’s set to change the way developers work with their computers and manage their coding tasks.

Filed Under: Guides, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.

Categories
News

Semantic Router superfast decision layer for LLMs and AI agents

Semantic Router superfast decision layer for LLMs and AI agents

In the rapidly evolving world of artificial intelligence, a new framework is enhancing the way we create and interact with chatbots and AI assistants. This innovative tool, known as the Semantic Router, is reshaping our expectations of digital conversations by offering a level of understanding and response accuracy that was previously unattainable. James Briggs explains a more about the Semantic Router system

“Semantic Router is a superfast decision layer for your LLMs and agents that integrates with LangChain, improves RAG, and supports OpenAI and Cohere. Rather than waiting for slow LLM generations to make tool-use decisions, we use the magic of semantic vector space to make those decisions — routing our requests using semantic meaning. This approach unlock incredibly fast agentic decision making, the ability to use literally millions of tools, and provide much greater steerability and AI safety using semantics.”

At its core, the Semantic Router serves as a sophisticated decision-making layer that works in tandem with language models. Its primary function is to guide chatbots in delivering prompt and pertinent answers to user inquiries. By navigating through a semantic vector space, the router is able to align user questions with the most fitting predefined responses. This process significantly improves the reliability of the chatbot’s answers, ensuring that users receive the information they need without unnecessary delays or confusion.

The benefits of this technology are particularly evident in its ability to provide consistent and rapid responses. This is crucial for creating a smooth user experience, especially in environments where the performance of AI is under close scrutiny. Whether it’s for customer service, information retrieval, or casual conversation, the Semantic Router’s efficiency is a key factor in its success.

Semantic Router superfast LLM decision layer

Here are some other articles you may find of interest on the subject of large language models (LLMs)

Integrating the Semantic Router into existing chatbot systems is surprisingly straightforward. The initial setup involves initializing an embedding model and configuring API keys. Once integrated, the router employs various conversational routes to maintain the relevance and flow of the dialogue. These routes include protective measures to prevent the conversation from veering off-topic and chitchat paths that allow for a more natural and engaging interaction.

The framework is designed with both standard and hybrid route layers to cater to different conversational needs. Standard layers are responsible for handling routine exchanges, while hybrid layers offer a blend of predefined and dynamic responses. This combination allows for more intricate and flexible conversations that can adapt to the complexities of human dialogue.

The introduction of the Semantic Router has had a profound impact on the behavior of chatbots, making them appear more controlled, reliable, and, ultimately, more human-like in their interactions. Users can now expect a level of conversational competence that mirrors human conversation more closely than ever before. Another significant aspect of this AI framework is its open-source nature. By inviting community participation and collaboration, the framework benefits from a diverse range of insights and contributions. This collective approach is essential for the continuous improvement of the technology and the introduction of new features, such as dynamic routing and hybrid layers.

The Semantic Router framework is poised to elevate the standard of AI-assisted communication and more information is available over on the official GitHub repository. By laying a solid foundation for chatbots and AI agents to deliver precise, reliable, and context-aware responses, this technology is enhancing the way we interact with digital assistants. As we continue to integrate AI into our daily lives, tools like the Semantic Router ensure that our conversations with machines become more natural and effective, bridging the gap between human and artificial communication.

Filed Under: Technology News, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.

Categories
News

The best tiny, small and compact LLMs currently available

The best tiny and compact LLMs to download today

AI models the foundation of the current boom in artificial intelligence are under constant development.  Within this dynamic realm, smaller AI models or compact Large Language Models (LLMs) have emerged as a notable trend. These models, which include Deep Seek Coder, TinyLlama, and Microsoft’s Phi-2, are designed to be both efficient and adaptable, making them suitable for a wide range of applications. They are particularly appealing for their ability to run on standard consumer hardware, which opens up possibilities for users who need advanced language processing capabilities without the high costs associated with larger, more complex models.

Deep Seek Coder, with its 1.3 billion parameters, and Microsoft’s F2, which boasts 2.7 billion parameters, are at the forefront of this movement. They represent a sweet spot in the AI world, where they are small enough to be manageable but still powerful enough to handle demanding tasks. This balance is crucial for those who want to leverage AI technology without investing in expensive infrastructure.

One of the key advantages of these compact LLMs is their ability to be customized for specific tasks. Techniques such as low-rank adaptation, or Lora, are instrumental in this process. They enable users to fine-tune the models to their unique requirements while keeping the number of trainable parameters relatively low. This means that you can achieve high performance without the need for the extensive computational resources that larger models demand.

Best compact large language models (LLMs) currently available

Here are some other articles you may find of interest on the subject of large language model (LLM) technologies :

When it comes to specific tasks like function calling, compact LLMs can be quite proficient. However, they are not without their challenges. For instance, the custom model Tris Tiny, which also has 1.3 billion parameters, demonstrates that while these models can handle function calling, they may struggle with more complex operations such as chain function calling. Moreover, these models have a tendency to generate verbose responses, which may not be ideal in every situation.

Another factor that can impact the performance of compact LLMs is quantization, particularly in tasks involving function calling. When Open Chat models are subjected to different levels of quantization, they exhibit varying degrees of efficiency and accuracy. Finding the right balance is essential to ensure that the model remains both responsive and precise.

Despite these hurdles, compact LLMs are still a viable choice for many applications. To make the most of these models, it is crucial to employ effective fine-tuning and inference techniques. This includes adjusting the number of trainable parameters and using helper text to guide the model’s responses, which helps to ensure that the outputs are relevant and concise.

Selecting the right compact LLM for your project is a critical decision. Whether you opt for Deep Seek Coder, Tiny Llama, or Microsoft’s F2, understanding their capabilities and how to fine-tune them is essential. With a thoughtful approach, these compact LLMs can provide efficient and potent language processing tools, becoming invaluable components in your AI arsenal.

Microsoft’s Phi-2

Phi-2 is a Transformer with 2.7 billion parameters. It was trained using the same data sources as Phi-1.5, augmented with a new data source that consists of various NLP synthetic texts and filtered websites (for safety and educational value). When assessed against benchmarks testing common sense, language understanding, and logical reasoning, Phi-2 showcased a nearly state-of-the-art performance among models with less than 13 billion parameters.

Trelis Tiny

Trelis Tiny, a model with 1.3 billion parameters, stands out for its ability to perform function calling, a feature crucial for dynamic and interactive tasks. It boasts a rapid token generation speed, a vital aspect for efficiency, regardless of whether it’s operated locally or hosted remotely.

Interested users can acquire access to this model, which also guarantees them updates on future enhancements made to the Tiny model in the same repository. Notably, the function metadata format aligns with that used by OpenAI, ensuring compatibility and ease of integration. The model is deemed fit for commercial applications, broadening its usability across various business contexts.

DeepSeek Coder 1.3B

Deepseek Coder is composed of a series of code language models, each trained from scratch on 2T tokens, with a composition of 87% code and 13% natural language in both English and Chinese. Various sizes of the code model, ranging from 1B to 33B versions have been made available.

Each model is pre-trained on project-level code corpus by employing a window size of 16K and a extra fill-in-the-blank task, to support project-level code completion and infilling. For coding capabilities, Deepseek Coder achieves state-of-the-art performance among open-source code models on multiple programming languages and various benchmarks.

TinyLlama-1.1B

The TinyLlama project aimed to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, the team achieved this within a span of “just” 90 days using 16 A100-40G GPUs. The training has started on 2023-09-01.

The potential of compact LLMs is vast, and as the technology continues to evolve, we can expect to see even more sophisticated and accessible models. These advancements will likely lead to a broader adoption of AI across various industries, enabling more people to harness the power of machine learning for their projects. As we navigate this exciting field, staying informed about the latest developments and understanding how to effectively implement these models will be key to success.

Filed Under: Guides, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.

Categories
News

Train LLMs faster using Unsloth x30 times faster

Unsloth makes training LLMs and AI models x30 faster

Training large language models is a critical part of AI development, but it’s also a process that can take a lot of time and use up a lot of computing power. That’s where Unsloth by Moonshot comes in allowing you to train LLMs faster. They’ve created a new software package that’s making a big advancements in how quickly and efficiently these models can be trained. It’s designed to work with a variety of graphics processing units (GPUs) from top companies like NVIDIA, Intel, and AMD.

Features of Unsloth AI training

  • 30x faster. Alpaca takes 3 hours instead of 85.
  • 60% less memory usage, allowing 6x larger batches.
  • 0% loss in accuracy or +20% increased accuracy with our Max offering.
  • No need for new hardware – only software changes.
  • Supports NVIDIA, Intel and AMD GPUs with our Max offering.
  • Manual autograd and chained matrix multiplication optimizations.
  • Rewrote all kernels in OpenAI’s Triton language.
  • Flash Attention via xformers and Tri Dao’s implementation.
  • Free open source version makes finetuning 2x faster with 50% less memory.

Imagine being able to shrink an 85-hour training session down to just 3 hours. Or train your own ChatGPT in 24 hrs instead of 30 days. That’s the kind of improvement we’re talking about with Unsloth AI’s software. It’s not just a small step forward; it’s a huge leap that can make your work 30 times faster. Plus, the software can cut memory usage by 60%, which means you can run bigger batches of data at once. This kind of optimization lets you do more with the computing resources you have.

Unsloth AI’s software is packed with advanced features that help it perform so well. It includes a custom autograd implementation in PyTorch, mathematical optimizations, and kernels that have been reworked using OpenAI’s Triton language. The software also uses something called the Flash attention mechanism to improve its capabilities even more.

Train your LLM is an AI models faster using Unsloth

Here are some other articles you may find of interest on the subject of training large language models :

Train LLMs faster

No matter what your budget or needs are, Unsloth AI has a plan that should work for you. They offer different tiers, including a free plan, Onslot Pro, and Onslot Max. Each one gives you different levels of training acceleration and memory usage improvements.

When you compare Unsloth AI’s software to other frameworks out there, like the ones from Hugging Face’s Transformers, it really stands out. It’s particularly good when you’re working with large datasets, offering impressive speed advantages that make it a great choice for AI developers.

Use a T4 GPU or Google Colab

One of the best things about this software is that it’s designed to work with the kind of hardware that many developers already have. For example, you can train your models on a standard T4 GPU, which means you can use platforms like Google Colab without needing to invest in expensive, specialized hardware.

The T4 GPU, developed by NVIDIA, is part of their Tesla series of GPUs, which are specifically designed for data center and AI workloads. It’s important to outline its key features and intended use:

  • Architecture: The T4 is based on the Turing architecture, which is also used in NVIDIA’s gaming and professional visualization products. This architecture is known for its efficiency and performance, particularly in AI and machine learning tasks.
  • AI and Machine Learning: One of the primary applications of the T4 is in AI and machine learning. It supports various AI frameworks and provides acceleration for AI inference and training tasks. Its architecture is optimized for these operations, making it a popular choice in environments where AI workloads are significant.
  • Tensor Cores: A distinctive feature of the T4 (and Turing architecture) is its Tensor Cores. These are specialized cores designed to accelerate deep learning tasks. They are highly efficient at performing the matrix operations which are common in neural network calculations.
  • Energy Efficiency: The T4 is notable for its energy efficiency. It delivers a significant amount of computing power for its size and power consumption, making it an attractive option for data centers where energy efficiency is a priority.
  • Versatility: Besides AI and ML, the T4 is also used for other data center workloads like graphics rendering, video processing, and general-purpose computing (thanks to its CUDA cores).
  • Form Factor and Deployment: The T4’s compact, low-profile design allows it to fit into a wide range of server and data center configurations. This flexibility is beneficial for businesses looking to integrate GPU acceleration without needing specialized hardware setups.
  • Multi-Precision Computing: The T4 supports mixed-precision computing, which allows it to adjust its precision level to optimize for performance or accuracy as needed. This is particularly useful in AI workloads where different stages of neural network training and inference can benefit from different levels of precision.

Adding Unsloth AI’s software to your current projects is easy. You won’t have to make big changes to your codebase, and the software is user-friendly when it comes to inputting data. It supports the Alpaca prompt template or format, so you can get started without any hassle. After you’ve finished training your models with Unsloth AI’s software, you can refine them and then use other packages for inference and deployment. This creates a smooth workflow that can save you money and help you develop your AI projects more quickly.

Unsloth AI’s new software package is a powerful tool that’s changing the way developers train large language models. It significantly reduces training time and memory requirements, works with a wide range of GPUs, and is easy to integrate into your existing projects. With this software, you can speed up your AI development and stay ahead in the competitive world of AI. Unsloth AI is helping to usher in a new era of AI model training, and it’s an exciting time to be a part of this field.

Filed Under: Guides, Top News





Latest timeswonderful Deals

Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.