2021 is shaping up to be a great year for developers, but not so much for companies hiring them, with 61% of HR professionals believing this will be their biggest challenge in the year ahead. Recruiters expect a shortage of qualified programmers in 2021. fizkes / iStock Hiring managers worry that they’ll struggle to recruit […]
Pioneering scientists like Rothermel dealt with this intractable problem by ignoring it. Instead, they searched for factors, such as wind speed and slope, that could help them predict a fire’s next move in real time. Looking back, Finney says, it’s a miracle that Rothermel’s equations work for wildfires at all. There’s the sheer difference in […]
From smart toilets and disinfecting robots to transparent OLED displays and sleep tech, CES 2021 was a showcase for the latest innovations in consumer and enterprise technology. CES 2021 is a wrap. And although this year’s all-digital event was a significantly different experience from past shows, there was plenty of innovative tech on display. TechRepublic’s […]
Commentary: Rasa isn’t the only open source approach to natural language processing, but its large community suggests it’s doing something right.
You want a conversational artificial intelligence (AI) platform? No problem–you just need to choose one. Microsoft has one (LUIS). So does Google (Dialogflow). AWS? Yep. (Lex.) But don’t stop now: There are hundreds of options (from Kore.ai to SAP to Cisco’s MindMeld to etc. etc.).
I’m not in a good position to gauge the utility of Rasa’s code. What I find fascinating is just how much community the project has attracted. This may well speak to the efficacy of Rasa’s open source approach, but also to how mainstream conversational AI has become, or soon will be.
More than open source
Rasa’s team could be right about the need to make conversational AI an open source problem, yet wrong in its approach. After all, there are plenty of other open source conversation AI platforms. Rasa isn’t the first to figure out that developers increasingly prefer open source infrastructure.
Rasa’s community is interested in customizing NLP. This is one reason it attracts more than 10,000 people to its Rasa community forum. It’s also why Rasa has over 500 contributors to the project. When I expressed surprise that there would be a large population of developers with aptitude to be able to contribute meaningful code to something like Rasa, Alan Nichol, Rasa’s co-founder and CTO, told me that it’s “pretty much the opposite” of what I suggested. No, not all of these will be experts in NLP, he continued, but valuable contributions might be integrations with various messaging platforms, or extensions of Rasa’s functionality to support new APIs that chat platforms might use.
Even for those who don’t contribute back, it’s important that Rasa be open source, Nichol noted:
[C]onversational AI is one of the [areas of software] where you benefit most [from open source]. The fact that you can customize it to make it your own, even if those aren’t necessarily changes that you push upstream, it’s extremely valuable. Much more so than the amount of people who might write a custom something inside of MongoDB or something like that. The amount of people who may write a custom NLP component to do sentiment analysis or do some categorization of their users, or just to tweak some hyper-parameters, use word embeddings that they trained on their own company’s corpus, all those kinds of things. There are lots and lots of ways that people customize the software.
The real competition for something like Rasa is customers who might roll their own conversational AI bot, perhaps using TensorFlow. Rasa is built on TensorFlow, and for a sufficiently skilled team, they could bypass Rasa and work directly on the lower-level TensorFlow. Rasa’s bet is that most companies won’t have the expertise or patience to do this.
They’ll also likely be looking for something ready for production, rather than projects like Uber’s Plato or Facebook’s ParlAI, which tends to be geared toward researchers. For Rasa, it has been important to merge language understanding and dialogue models into one end-to-end system, so that when you have messages that don’t neatly fit into a schema, the AI learns, rather than breaking down (“takes that user’s utterance and transforms it into a vector of floating point numbers into a continuous representation,” is the more geeky explanation that Nichols offered).
The good news is that you don’t have to take my word for it–or the word of Nichols or Dunn. It’s open source. You can check it out on GitHub, customize it to meet your needs and, hopefully, submit a pull request to improve it.
Disclosure: I work for AWS, but the views expressed herein are mine.
Data, Analytics and AI Newsletter
Learn the latest news and best practices about data science, big data analytics, and artificial intelligence. Delivered Mondays