RobjHunt
Regular
You can't be serious?? The same reason it shoot up all the other times. But Dicklebro will tell us all that BrainChip is not one of the 5 shares that he/they will suggest
BRN Discussion Ongoing
- Thread starter TechGirl
- Start date
Smoothsailing
Regular
Brilliant Go Brainchip
NVIDIA and ither cloud providers operate their version of AI at the Edge via the Cloud which is really 2nd rate. True AI at the Edge is Neuromorphic based. At the moment BRN is the only commercial provider.
Check the presentation out. Sean has a couple of slides and verbal explanations.
Sean pretty much trashes NVIDIA as a true AI at the Edge provider.
Cloud provides traditional AI services extremely well - At the moment that is pretty much where almost all the business is. True AI at the Edge is real time on the chip. There is a lot of confusion about this.
Real time AI at the Edge is expected to grow exponentially.
Pom down under
Top 20
And the reason we don't manufacture an Akida2 chip ourselves is because we made the decision 4 years ago to be IP only and the reason we're sticking with it is because.....
CHIPS
Regular
Fullmoonfever
Top 20
I see Unigen were just at DCD Connect in NY.
Hopefully they might have had some bakery goods to show off as well
Unigen Corporation
5,732 followers
1w
Join Unigen's Product Marketing Director, Oliver Baltuch, at DCD Connect 2024! Visit us at booth #59 at the New York Marriott Marquis on March 18-19 to explore Unigen's latest data center products, featuring our Mercier Storage Accelerator. Feel free to send Oliver a message on LinkedIn to arrange a meet-up. #DCDConnect
Hopefully they might have had some bakery goods to show off as well
Unigen Corporation
5,732 followers
1w
Join Unigen's Product Marketing Director, Oliver Baltuch, at DCD Connect 2024! Visit us at booth #59 at the New York Marriott Marquis on March 18-19 to explore Unigen's latest data center products, featuring our Mercier Storage Accelerator. Feel free to send Oliver a message on LinkedIn to arrange a meet-up. #DCDConnect
I am constantly describing our chip-to-cloud architecture as the future of our vehicles. | Magnus Östberg
I am constantly describing our chip-to-cloud architecture as the future of our vehicles. Since the terminology isn’t very meaningful without context, let’s define it then break down its meaning piece by piece for those who might be new to our community. Our chip-to-cloud architecture is a...
Frangipani
Top 20
Is this possibly an unintended reveal of the Austrian Space Agency’s (so far) super secret involvement in that USAFA experiment onboard the ISS?!
🛰
A tribute to the late Johann ‘Hans’ Hölzel, better known by his stage name Falco, whose 1985 hit Rock Me Amadeus was the only German-speaking song to date that ever reached No. 1 on the Billboard Hot 100 chart?
Imagine a day in 2019 or 2020 - the exact date is clouded in secrecy.
Suddenly, the United States Air Force Academy phone rings. A cadet picks up.
“Vienna Calling, here. Der Kommissar. We would like to suggest a code name for that future neuromorphic camera experiment in space, to honour the memory of Austria’s biggest pop star! His stage name would be the perfect choice, as he even sang about space tech back in 1988”.
OMG, this missing letter ‘n’ and the ensuing chain of thoughts just triggered heaps of memories - from an open air Falco concert at the Bregenz floating opera stage (on Lake Constance), to controversial lyrics and boycotts by radio stations (native speakers of German may recall Jeanny) and mixtape nostalgia.
For those of you who weren’t teenagers (or their parents) in the 1980s, when cassette tapes were SOTA, the video below is a good explanation of the labour of love that went into these works of art, the creation of which took a lot of time and patience (as you had to sit in front of the radio for hours, hoping specific songs would be played) as well as intense prayers for radio DJs not to talk into the eagerly awaited songs’ intro or fade-out. Equally nerve-wrecking were family members spoiling the recording, both inadvertently and intentionally. Add in the time spent on hand-writing the song list and creating a cover to go along with the tape’s theme (in case there was one).
Besides making personalised playlists for self-use (to this day, I can’t bear parting with the testimony of my youth’s soundtrack), they were the ultimate musical gift for best friends, secret crushes, lovers etc. And apparently some people even went to great lengths in compiling songs as a poisoned parting gift (incidentally, Gift is the German word for poison
) for their ex-partner. So much reading between the lines. Truly a lost art in the age of YouTube, Spotify, Amazon and Apple Music.My Google search for some info on mixtapes for the clueless digital natives also surfaced this excellent write-up for those wanting to go down memory lane:
https://www.wsj.com/arts-culture/music/never-mind-spotifyremember-the-lost-art-of-the-mixtape-98d03060
Fullmoonfever
Top 20
View attachment 59586
Is this possibly an unintended reveal of the Austrian Space Agency’s (so far) super secret involvement in that USAFA experiment onboard the ISS?!
A tribute to the late Johann ‘Hans’ Hölzel, better known by his stage name Falco, whose 1985 hit Rock Me Amadeus was the only German-speaking song to date that ever reached No. 1 on the Billboard Hot 100 chart?
Imagine a day in 2019 or 2020 - the exact date is clouded in secrecy.
Suddenly, the United States Air Force Academy phone rings. A cadet picks up.
“Vienna Calling, here. Der Kommissar. We would like to suggest a code name for that future neuromorphic camera experiment in space, to honour the memory of Austria’s biggest pop star! His stage name would be the perfect choice, as he even sang about space tech back in 1988”.
View attachment 59584
OMG, this missing letter ‘n’ and the ensuing chain of thoughts just triggered heaps of memories - from an open air Falco concert at the Bregenz floating opera stage (on Lake Constance), to controversial lyrics and boycotts by radio stations (native speakers of German may recall Jeanny) and mixtape nostalgia.
For those of you who weren’t teenagers (or their parents) in the 1980s, when cassette tapes were SOTA, the video below is a good explanation of the labour of love that went into these works of art, the creation of which took a lot of time and patience (as you had to sit in front of the radio for hours, hoping specific songs would be played) as well as intense prayers for radio DJs not to talk into the eagerly awaited songs’ intro or fade-out. Equally nerve-wrecking were family members spoiling the recording, both inadvertently and intentionally. Add in the time spent on hand-writing the song list and creating a cover to go along with the tape’s theme (in case there was one).
Besides making personalised playlists for self-use (to this day, I can’t bear parting with the testimony of my youth’s soundtrack), they were the ultimate musical gift for best friends, secret crushes, lovers etc. And apparently some people even went to great lengths in compiling songs as a poisoned parting gift (incidentally, Gift is the German word for poison
My Google search for some info on mixtapes for the clueless digital natives also surfaced this excellent write-up for those wanting to go down memory lane:
https://www.wsj.com/arts-culture/music/never-mind-spotifyremember-the-lost-art-of-the-mixtape-98d03060
80s teen here....what a decade to be a teen it was.
Incredible memories...thanks for the post.
You made me look Nena up as I thought surely she got to #1 too but nope... #2
I still have about 4 shoeboxes of favourite tapes, including some mix tapes from the late 70s / 80s, some off the radio at the time haha
They go with my fully boxed (working) Commodore 64, with all the peripherals and about 80 floppys (that's disk's for you digital natives with filthy minds
) and datasettes and my first Motorola Microtac mobile and charger etc.Hoarder much
Frangipani
Top 20
80s teen here....what a decade to be a teen it was.
Incredible memories...thanks for the post.
You made me look Nena up as I thought surely she got to #1 too but nope... #2
I still have about 4 shoeboxes of favourite tapes, including some mix tapes from the late 70s / 80s, some off the radio at the time haha
They go with my fully boxed (working) Commodore 64, with all the peripherals and about 80 floppys (that's disk's for you digital natives with filthy minds
Hoarder much
Yeah, same here
- in fact, I had to double-check Nena’s ranking as well, as I had also assumed her 
x 33 would have surely floated above everything else for at least a week!Not sure if our C64 is still around at my parents’ place - I wasn’t into taking it apart and putting it back together or into coding, but I vividly remember playing Summer and Winter Games with my siblings in the 80s! But I do still have floppy discs, my beloved Walkman cassette player (one of the best Christmas gifts ever!), my first Nokia phone etc., so I can relate!
We should organise a virtual 80s party one day!
Apparently our teenage decade is trending again! 
Humble Genius
Regular
View attachment 59586
Is this possibly an unintended reveal of the Austrian Space Agency’s (so far) super secret involvement in that USAFA experiment onboard the ISS?!
A tribute to the late Johann ‘Hans’ Hölzel, better known by his stage name Falco, whose 1985 hit Rock Me Amadeus was the only German-speaking song to date that ever reached No. 1 on the Billboard Hot 100 chart?
Imagine a day in 2019 or 2020 - the exact date is clouded in secrecy.
Suddenly, the United States Air Force Academy phone rings. A cadet picks up.
“Vienna Calling, here. Der Kommissar. We would like to suggest a code name for that future neuromorphic camera experiment in space, to honour the memory of Austria’s biggest pop star! His stage name would be the perfect choice, as he even sang about space tech back in 1988”.
View attachment 59584
OMG, this missing letter ‘n’ and the ensuing chain of thoughts just triggered heaps of memories - from an open air Falco concert at the Bregenz floating opera stage (on Lake Constance), to controversial lyrics and boycotts by radio stations (native speakers of German may recall Jeanny) and mixtape nostalgia.
For those of you who weren’t teenagers (or their parents) in the 1980s, when cassette tapes were SOTA, the video below is a good explanation of the labour of love that went into these works of art, the creation of which took a lot of time and patience (as you had to sit in front of the radio for hours, hoping specific songs would be played) as well as intense prayers for radio DJs not to talk into the eagerly awaited songs’ intro or fade-out. Equally nerve-wrecking were family members spoiling the recording, both inadvertently and intentionally. Add in the time spent on hand-writing the song list and creating a cover to go along with the tape’s theme (in case there was one).
Besides making personalised playlists for self-use (to this day, I can’t bear parting with the testimony of my youth’s soundtrack), they were the ultimate musical gift for best friends, secret crushes, lovers etc. And apparently some people even went to great lengths in compiling songs as a poisoned parting gift (incidentally, Gift is the German word for poison
My Google search for some info on mixtapes for the clueless digital natives also surfaced this excellent write-up for those wanting to go down memory lane:
https://www.wsj.com/arts-culture/music/never-mind-spotifyremember-the-lost-art-of-the-mixtape-98d03060
For our German friends... a trip down memory lane...
Sorry, is this got anything to do with us?
The Pope
Regular
Are you looking for info to write a article tomorrow
Apology accepted
Frangipani
Top 20
The following LinkedIn newsletter about how future Small Language Models (SLMs) can potentially aid healthcare professionals and patients (although the actual title doesn’t really make sense, since ChatGPT is based on an LLM 
) as well as the comments reminded me of Sally Ward-Foxton’s December 2023 EETimes podcast with Nandan Nayampally. From 20:35 min onwards, he draws attention to the significant advantage offered by personalised healthcare monitoring, when he relates to a medical issue his wife had had, which was initially overlooked, as her parameters seemed normal compared to the general population, but in fact weren’t.
“But the two big new things that we’ve added are, firstly, I’ll say, vision transformers is not new, but an efficient encoding that can be put in small footprints is new. And the other one that has been garnering us a lot of attention is the temporal, event based neural nets. And the idea of the temporal event based neural nets is the ability to recurrent layers more efficiently, and time series data or sequential data analysis or analytics much more capably. And so that really changes the way Akida can support much wider ranges of applications, from high end video object detection in real time on a portable device, to potentially, you know, healthcare monitoring, on patient, which is managed and secure and personalized. So the thing we haven’t talked yet, which is common across both generations, is our ability to learn on device.
Now this is not re-training, because that’s a pretty complex process, and expensive process. But here, we can take the learning that has been done or features extracted from training, and we can extend it on device with a more flexible final layer. So if you know that the model can recognize faces with one shot or multi-shot learning, we can now say, hey, this is Sally’s face, or this is Nandan’s face. You don’t want two thousand new faces to train, but for most of these devices, it’s really the owner and maybe the family, and similarly from healthcare, it is fundamentally interesting, because even though today’s healthcare deals with statistics, and if you are on the edge of that statistic, are you normal, are you not? What does that mean for me, right? My BP is X. My blood pressure is X. It could fall in the formal range, and I get treated like everybody in that range, but actually for me it may mean something different. And I have personal experience with that, especially my wife’s health, that was misunderstood, because she was on the edge of a range that was considered normal, and hence treatment took a lot longer to come, because they didn’t feel she was out of bounds, when if it is personalized, they would have known that whatever was happening to her was out of bounds, and we would have moved to more of a preventative rather than a post facto and hence much more painful treatment.”
www.linkedin.com
Using ChatGPT Offline: How Small Language Models Can Aid Healthcare Professionals
924 articles
March 21, 2024
By now, you might have come across the term large language models (LLMs), which is a type of generative artificial intelligence (GenAI). If not, you have likely encountered GenAI applications that are based on LLMs. This includes the likes of ChatGPT, Google Bard and Microsoft Copilot. While such models have proved useful, in the healthcare setting, they come with new sets of regulatory, ethical and privacy concerns.
Recently, another type of language model has been gaining attention in the GenAI field: the small language model (SLM). It even holds the promise of addressing some of the challenges with integrating LLMs in healthcare. In this article, we will introduce SLMs, compare them to LLMs and explore their healthcare potential.
SLMs are termed as “small” as they are trained on relatively small amounts of data and have a relatively small number of parameters. Parameters here refer to variables that define the model’s structure and behavior.
Emphasis should be paid on relatively as SLMs still involve millions or even billions of parameters. However, this range of parameters in SLM is “small” in comparison to LLMs and we’ll consider the differences between these models in the next section.
This difference in architecture is reflected in the resources required to run the different types of models. LLMs require significant computing resources from servers to storage. Such needs trickle down to the huge costs associated with running such models; and are thus not accessible or even feasible to every organisation. In comparison, SLMs can be small enough to run offline on a phone while bearing significantly less operational costs.
“Small language models can make AI more accessible due to their size and affordability,” says Sebastien Bubeck, who leads the Machine Learning Foundations group at Microsoft Research. “At the same time, we’re discovering new ways to make them as powerful as large language models.”
While a complex model with more parameters can be more powerful, SLMs can still have an edge over LLMs. By being trained on smaller and more specialized datasets, SLMs can be more efficient for specific cases, even if this means having a narrower scope than LLMs.
This can even lead to SLMs to outperform LLMs in certain cases. Microsoft exemplified this with their Phi-2 SLM, which performed better in coding and maths tasks compared to the Llama-2 LLM which is 25 times larger than Phi-2.
While SLM tools have yet to be publicly deployed in healthcare settings, their advantages indicate that it is only a matter of time until this happens. Big tech companies are actively working on such models. Microsoft researchers have developed and released two SLMs, namely Phi and Orca. French AI startup Mistral has released Mixtral-8x7B, which can run on a single computer (with plenty of RAM). Google has Gemini Nano, which can run on smartphones.
However, SLM tools will also bring about their respective concerns when they eventually roll out for healthcare purposes. They will have to adhere to similar regulations we propose for LLMs in order to ensure their safe and ethical applications. As Microsoft lists SLMs as one of the big AI trends to watch in 2024, it might be worthwhile for the healthcare community to do the same.
Report this
Bertalan Meskó, MD, PhDBertalan Meskó, MD, PhD
Director of The Medical Futurist Institute (Keynote Speaker, Researcher, Author & Futurist)Director of The Medical Futurist Institute (Keynote Speaker, Researcher, Author & Futurist)
Published • 4h
924 articlesFollow
Small language models (SLMs) are gaining attention in the generative artificial intelligence field. They have relatively small number of parameters, and can, for example, run on an average mobile phone, without internet access.
So let's ask the obvious question: How could they benefit healthcare?
Let's see!
75
) as well as the comments reminded me of Sally Ward-Foxton’s December 2023 EETimes podcast with Nandan Nayampally. From 20:35 min onwards, he draws attention to the significant advantage offered by personalised healthcare monitoring, when he relates to a medical issue his wife had had, which was initially overlooked, as her parameters seemed normal compared to the general population, but in fact weren’t.“But the two big new things that we’ve added are, firstly, I’ll say, vision transformers is not new, but an efficient encoding that can be put in small footprints is new. And the other one that has been garnering us a lot of attention is the temporal, event based neural nets. And the idea of the temporal event based neural nets is the ability to recurrent layers more efficiently, and time series data or sequential data analysis or analytics much more capably. And so that really changes the way Akida can support much wider ranges of applications, from high end video object detection in real time on a portable device, to potentially, you know, healthcare monitoring, on patient, which is managed and secure and personalized. So the thing we haven’t talked yet, which is common across both generations, is our ability to learn on device.
Now this is not re-training, because that’s a pretty complex process, and expensive process. But here, we can take the learning that has been done or features extracted from training, and we can extend it on device with a more flexible final layer. So if you know that the model can recognize faces with one shot or multi-shot learning, we can now say, hey, this is Sally’s face, or this is Nandan’s face. You don’t want two thousand new faces to train, but for most of these devices, it’s really the owner and maybe the family, and similarly from healthcare, it is fundamentally interesting, because even though today’s healthcare deals with statistics, and if you are on the edge of that statistic, are you normal, are you not? What does that mean for me, right? My BP is X. My blood pressure is X. It could fall in the formal range, and I get treated like everybody in that range, but actually for me it may mean something different. And I have personal experience with that, especially my wife’s health, that was misunderstood, because she was on the edge of a range that was considered normal, and hence treatment took a lot longer to come, because they didn’t feel she was out of bounds, when if it is personalized, they would have known that whatever was happening to her was out of bounds, and we would have moved to more of a preventative rather than a post facto and hence much more painful treatment.”
Using ChatGPT Offline: How Small Language Models Can Aid Healthcare Professionals
By now, you might have come across the term large language models (LLMs), which is a type of generative artificial intelligence (GenAI). If not, you have likely encountered GenAI applications that are based on LLMs.
Using ChatGPT Offline: How Small Language Models Can Aid Healthcare Professionals
Bertalan Meskó, MD, PhD
Director of The Medical Futurist Institute (Keynote Speaker, Researcher, Author & Futurist)924 articles
March 21, 2024
By now, you might have come across the term large language models (LLMs), which is a type of generative artificial intelligence (GenAI). If not, you have likely encountered GenAI applications that are based on LLMs. This includes the likes of ChatGPT, Google Bard and Microsoft Copilot. While such models have proved useful, in the healthcare setting, they come with new sets of regulatory, ethical and privacy concerns.
Recently, another type of language model has been gaining attention in the GenAI field: the small language model (SLM). It even holds the promise of addressing some of the challenges with integrating LLMs in healthcare. In this article, we will introduce SLMs, compare them to LLMs and explore their healthcare potential.
What are SLMs?
Like LLMs, SLMs are a type of GenAI and operate in similar ways. This means that they rely on neural networks to learn patterns from language in text to produce new text of their own.SLMs are termed as “small” as they are trained on relatively small amounts of data and have a relatively small number of parameters. Parameters here refer to variables that define the model’s structure and behavior.
Emphasis should be paid on relatively as SLMs still involve millions or even billions of parameters. However, this range of parameters in SLM is “small” in comparison to LLMs and we’ll consider the differences between these models in the next section.
SLM and LLM: what’s the difference?
While there is no clear threshold for SLMs, they usually tend to have between a hundred million to tens of billions of parameters. While still large numbers, this range is small compared to the number of parameters LLMs possess which can reach hundreds of billions. Consider OpenAI’s GPT-3: this LLM has 175 billion parameters, and GPT-4 is believed to have about a trillion parameters. In comparison, Microsoft recently introduced Phi-2, an SLM developed by the company’s researchers with 2.7 billion parameters.This difference in architecture is reflected in the resources required to run the different types of models. LLMs require significant computing resources from servers to storage. Such needs trickle down to the huge costs associated with running such models; and are thus not accessible or even feasible to every organisation. In comparison, SLMs can be small enough to run offline on a phone while bearing significantly less operational costs.
“Small language models can make AI more accessible due to their size and affordability,” says Sebastien Bubeck, who leads the Machine Learning Foundations group at Microsoft Research. “At the same time, we’re discovering new ways to make them as powerful as large language models.”
While a complex model with more parameters can be more powerful, SLMs can still have an edge over LLMs. By being trained on smaller and more specialized datasets, SLMs can be more efficient for specific cases, even if this means having a narrower scope than LLMs.
This can even lead to SLMs to outperform LLMs in certain cases. Microsoft exemplified this with their Phi-2 SLM, which performed better in coding and maths tasks compared to the Llama-2 LLM which is 25 times larger than Phi-2.
SLMs’ potentials in healthcare
By focusing on curated, high-quality data and requiring less computational and financial resources, SLMs are particularly apt for healthcare uses. While GenAI which is based on SLMs has not been publicly released, we can contemplate some of the technology’s potential.
1. Personalised patient journey
By training an SLM-based GenAI on relatively small but high-quality datasets, patients can receive a personalized healthcare experience. This can be achieved by developing a chatbot that focuses on a specific condition and can provide patients with educational materials and recommendations specific to their conditions. With such a tool, each patient could even have a personal, artificial doctor’s assistant that guides them during their patient journey.
2. Affordable generative AI
By requiring fewer resources to train and run, SLMs are more affordable than their LLM counterparts. Such models could be deployed without the need for costly infrastructure such as specialised hardware and cloud services. Through such increased accessibility, more healthcare institutions could benefit from GenAI and further tune the technology to their individual needs, without compromising on efficiency.
3. Improved AI transparency
Thanks to their simpler architecture, SLM outputs are more interpretable and thus more transparent. Transparency over such AI models is further enhanced by the ability to better control the training data to address biases and be more reflective of the population it is assisting. This can further help in building trust in such AI tools.While SLM tools have yet to be publicly deployed in healthcare settings, their advantages indicate that it is only a matter of time until this happens. Big tech companies are actively working on such models. Microsoft researchers have developed and released two SLMs, namely Phi and Orca. French AI startup Mistral has released Mixtral-8x7B, which can run on a single computer (with plenty of RAM). Google has Gemini Nano, which can run on smartphones.
However, SLM tools will also bring about their respective concerns when they eventually roll out for healthcare purposes. They will have to adhere to similar regulations we propose for LLMs in order to ensure their safe and ethical applications. As Microsoft lists SLMs as one of the big AI trends to watch in 2024, it might be worthwhile for the healthcare community to do the same.
Report this
Published by
Bertalan Meskó, MD, PhDBertalan Meskó, MD, PhD
Director of The Medical Futurist Institute (Keynote Speaker, Researcher, Author & Futurist)Director of The Medical Futurist Institute (Keynote Speaker, Researcher, Author & Futurist)
Published • 4h
924 articlesFollow
Small language models (SLMs) are gaining attention in the generative artificial intelligence field. They have relatively small number of parameters, and can, for example, run on an average mobile phone, without internet access.
So let's ask the obvious question: How could they benefit healthcare?
Let's see!
Boab
I wish I could paint like Vincent
This makes a lot of sense. No need for those LLM's when you can have a specialised SLM to suit your particular need with no need for the cloud.The following LinkedIn newsletter about how future Small Language Models (SLMs) can potentially aid healthcare professionals and patients (although the actual title doesn’t really make sense, since ChatGPT is based on an LLM
“But the two big new things that we’ve added are, firstly, I’ll say, vision transformers is not new, but an efficient encoding that can be put in small footprints is new. And the other one that has been garnering us a lot of attention is the temporal, event based neural nets. And the idea of the temporal event based neural nets is the ability to recurrent layers more efficiently, and time series data or sequential data analysis or analytics much more capably. And so that really changes the way Akida can support much wider ranges of applications, from high end video object detection in real time on a portable device, to potentially, you know, healthcare monitoring, on patient, which is managed and secure and personalized. So the thing we haven’t talked yet, which is common across both generations, is our ability to learn on device.
Now this is not re-training, because that’s a pretty complex process, and expensive process. But here, we can take the learning that has been done or features extracted from training, and we can extend it on device with a more flexible final layer. So if you know that the model can recognize faces with one shot or multi-shot learning, we can now say, hey, this is Sally’s face, or this is Nandan’s face. You don’t want two thousand new faces to train, but for most of these devices, it’s really the owner and maybe the family, and similarly from healthcare, it is fundamentally interesting, because even though today’s healthcare deals with statistics, and if you are on the edge of that statistic, are you normal, are you not? What does that mean for me, right? My BP is X. My blood pressure is X. It could fall in the formal range, and I get treated like everybody in that range, but actually for me it may mean something different. And I have personal experience with that, especially my wife’s health, that was misunderstood, because she was on the edge of a range that was considered normal, and hence treatment took a lot longer to come, because they didn’t feel she was out of bounds, when if it is personalized, they would have known that whatever was happening to her was out of bounds, and we would have moved to more of a preventative rather than a post facto and hence much more painful treatment.”
Using ChatGPT Offline: How Small Language Models Can Aid Healthcare Professionals
By now, you might have come across the term large language models (LLMs), which is a type of generative artificial intelligence (GenAI). If not, you have likely encountered GenAI applications that are based on LLMs.
Using ChatGPT Offline: How Small Language Models Can Aid Healthcare Professionals
Bertalan Meskó, MD, PhD
Director of The Medical Futurist Institute (Keynote Speaker, Researcher, Author & Futurist)
924 articles
March 21, 2024
By now, you might have come across the term large language models (LLMs), which is a type of generative artificial intelligence (GenAI). If not, you have likely encountered GenAI applications that are based on LLMs. This includes the likes of ChatGPT, Google Bard and Microsoft Copilot. While such models have proved useful, in the healthcare setting, they come with new sets of regulatory, ethical and privacy concerns.
Recently, another type of language model has been gaining attention in the GenAI field: the small language model (SLM). It even holds the promise of addressing some of the challenges with integrating LLMs in healthcare. In this article, we will introduce SLMs, compare them to LLMs and explore their healthcare potential.
What are SLMs?
Like LLMs, SLMs are a type of GenAI and operate in similar ways. This means that they rely on neural networks to learn patterns from language in text to produce new text of their own.
SLMs are termed as “small” as they are trained on relatively small amounts of data and have a relatively small number of parameters. Parameters here refer to variables that define the model’s structure and behavior.
Emphasis should be paid on relatively as SLMs still involve millions or even billions of parameters. However, this range of parameters in SLM is “small” in comparison to LLMs and we’ll consider the differences between these models in the next section.
SLM and LLM: what’s the difference?
While there is no clear threshold for SLMs, they usually tend to have between a hundred million to tens of billions of parameters. While still large numbers, this range is small compared to the number of parameters LLMs possess which can reach hundreds of billions. Consider OpenAI’s GPT-3: this LLM has 175 billion parameters, and GPT-4 is believed to have about a trillion parameters. In comparison, Microsoft recently introduced Phi-2, an SLM developed by the company’s researchers with 2.7 billion parameters.
This difference in architecture is reflected in the resources required to run the different types of models. LLMs require significant computing resources from servers to storage. Such needs trickle down to the huge costs associated with running such models; and are thus not accessible or even feasible to every organisation. In comparison, SLMs can be small enough to run offline on a phone while bearing significantly less operational costs.
“Small language models can make AI more accessible due to their size and affordability,” says Sebastien Bubeck, who leads the Machine Learning Foundations group at Microsoft Research. “At the same time, we’re discovering new ways to make them as powerful as large language models.”
While a complex model with more parameters can be more powerful, SLMs can still have an edge over LLMs. By being trained on smaller and more specialized datasets, SLMs can be more efficient for specific cases, even if this means having a narrower scope than LLMs.
This can even lead to SLMs to outperform LLMs in certain cases. Microsoft exemplified this with their Phi-2 SLM, which performed better in coding and maths tasks compared to the Llama-2 LLM which is 25 times larger than Phi-2.
SLMs’ potentials in healthcare
By focusing on curated, high-quality data and requiring less computational and financial resources, SLMs are particularly apt for healthcare uses. While GenAI which is based on SLMs has not been publicly released, we can contemplate some of the technology’s potential.
1. Personalised patient journey
By training an SLM-based GenAI on relatively small but high-quality datasets, patients can receive a personalized healthcare experience. This can be achieved by developing a chatbot that focuses on a specific condition and can provide patients with educational materials and recommendations specific to their conditions. With such a tool, each patient could even have a personal, artificial doctor’s assistant that guides them during their patient journey.
2. Affordable generative AI
By requiring fewer resources to train and run, SLMs are more affordable than their LLM counterparts. Such models could be deployed without the need for costly infrastructure such as specialised hardware and cloud services. Through such increased accessibility, more healthcare institutions could benefit from GenAI and further tune the technology to their individual needs, without compromising on efficiency.
3. Improved AI transparency
Thanks to their simpler architecture, SLM outputs are more interpretable and thus more transparent. Transparency over such AI models is further enhanced by the ability to better control the training data to address biases and be more reflective of the population it is assisting. This can further help in building trust in such AI tools.
While SLM tools have yet to be publicly deployed in healthcare settings, their advantages indicate that it is only a matter of time until this happens. Big tech companies are actively working on such models. Microsoft researchers have developed and released two SLMs, namely Phi and Orca. French AI startup Mistral has released Mixtral-8x7B, which can run on a single computer (with plenty of RAM). Google has Gemini Nano, which can run on smartphones.
However, SLM tools will also bring about their respective concerns when they eventually roll out for healthcare purposes. They will have to adhere to similar regulations we propose for LLMs in order to ensure their safe and ethical applications. As Microsoft lists SLMs as one of the big AI trends to watch in 2024, it might be worthwhile for the healthcare community to do the same.
Report this
Published by
Bertalan Meskó, MD, PhDBertalan Meskó, MD, PhD
Director of The Medical Futurist Institute (Keynote Speaker, Researcher, Author & Futurist)Director of The Medical Futurist Institute (Keynote Speaker, Researcher, Author & Futurist)
Published • 4h
924 articlesFollow
Small language models (SLMs) are gaining attention in the generative artificial intelligence field. They have relatively small number of parameters, and can, for example, run on an average mobile phone, without internet access.
So let's ask the obvious question: How could they benefit healthcare?
Let's see!
75
View attachment 59624
View attachment 59625
Love it.
Ethinvestor
Regular
Great info.. thx
Similar threads
