Amazon Q, one of the company’s most important new artificial intelligence products, has been available to the public since late April. Four months after its launch, Q is facing a host of teething problems.
Some of those concerns were shared in August in an internal Slack channel, where several Amazon Web Services employees expressed their opinions about how Q had performed during its early rollout, according to messages and an internal memo viewed by Business Insider.
They say Q lacks some of the features that rival products have, is sometimes expensive for users, and has trouble integrating with other software. Worse, Q risks losing customers to Microsoft’s Copilot.
“My take is that Q is suited for demos and very tightly controlled simple use cases at the moment,” one AWS employee wrote in the Slack channel.
Q’s early struggles illustrate the challenges facing enterprise software companies in the AI era. Buzz, hype, and hope have fueled intense competition that forces tech companies to rush new products to market, often with incomplete features, flawed internal tracking systems, and unexpected computational effects. Amazon’s competitors, such as Microsoft and Google, have also struggled with early AI launches.
For Amazon, it’s a new test of its AI prowess. The company hopes to generate billions of dollars in revenue from its AI services this year. But AWS’s AI chips, which compete with Nvidia’s GPUs, have seen slow adoption, while efforts to upgrade the Alexa voice assistant with ChatGPT-like technology have run into stumbling blocks.
“Not up to par” by Copilot
Internally, Amazon employees worry that Q’s shortcomings will prompt customers to turn to Microsoft’s Copilot.
According to Slack messages and a separate internal memo viewed by BI, a common customer complaint is Q’s inability to handle images embedded in PDF files. As a result, one AWS employee wrote in Slack that Q’s responses are “not up to par” with Copilot’s, and that customers should wait until AWS’s Re:Invent conference in December, when upgrades are expected.
Another issue is image creation. The same AWS employee said Q doesn’t support “multimodal capabilities,” or the ability to handle both text and image files, making it less appealing. One customer switched to Copilot after discovering Q couldn’t create a marketing campaign that included both an image and text, the employee said.
Some customers have complained about the high cost of integrating Q data, according to the internal memo and Slack messages. For example, one customer found that Amazon Q lacked the tools to choose which email content to ingest when integrating with Microsoft Exchange for enhanced email search. The initial cost estimate for that customer was about $400 per user, per inbox. Microsoft’s Copilot, meanwhile, has “out-of-the-box capabilities” to support that use case, the employee said.
“Is this product ready for mass adoption?” the person wrote on Slack.
A healthy culture of self-criticism
Some Amazon employees sounded the alarm ahead of Q’s launch, saying the release was “rushed,” with little testing and a heavy reliance on human reviewers.
It’s part of a broader culture at Amazon that encourages employees to report problems so they can be fixed more quickly. Other major technology deployments at the company have struggled early on but then gradually improved and found success. Amazon Q could be one of those products.
In an email to BI, AWS spokesperson Patrick Neighorn said the open discussions are part of Amazon’s culture that encourages employees to be “loudly self-critical” about their work. He added that Amazon Q is seeing strong growth across the board.
“Being self-critical is essential to earning customer trust, and we seek and value feedback from manufacturers,” Neighorn wrote. “We’re already seeing major customers like Bayer, Smartsheet, and National Australia Bank using Amazon Q today, and we’re continuing to use Amazon Q across our businesses, including the AWS commercial organization and Prime Video.”
“Fire drill”
Some AWS employees in the Slack channel pointed out the company’s ambitious marketing language for Q and asked whether the company should “tone it down” so customers don’t lose trust.
“This is a fire drill waiting to happen,” one AWS employee wrote on Slack. “All we can do is plan to pick up the pieces once marketing is done pinning our customers against the wall of what Q can’t do.”
Neighorn told BI that Amazon has clear pricing for Amazon Q and that customers can use the AWS pricing calculator to calculate Amazon Q fees. He added that Amazon Q gives customers control over what documents are indexed and how they are ingested.
“Amazon Q only launched in April, and we continue to add new features based on customer feedback,” Neighorn said. “Amazon Q has a growing customer base that uses Q for a variety of use cases.”
Matt Garman, CEO of AWS. Amazon
Disappointing initial sales data
Internal data also points to disappointing sales for the quarter.
According to internal data obtained by BI, part of AWS’s sales team, which numbers more than 3,000 people, failed to meet the Q sales target it was supposed to achieve by July. The data, which was tracked by Amazon’s top management team, showed that Q sales fell short of its targets in all regions, including the North American market.
Amazon has also struggled to keep up with sales of this new AI product.
Neighorn said the internal sales data obtained by BI came from “inaccurate preliminary numbers based on an incomplete methodology that we corrected weeks ago.” BI is not releasing specific sales numbers and targets from the internal data because of Amazon’s correction of the methodology.
“Customers are excited about Amazon Q, which has seen rapid customer adoption since its launch just four months ago and is already close to meeting our ambitious sales goals,” Neighorn said.
$260 million in efficiency gains
AWS CEO Matt Garman asked part of his sales team of about 6,500 employees to take a mandatory half-day training for Amazon Q earlier this year, according to an internal message seen by BI.
The training, designed to build the “confidence and skills” needed to sell Q, covered everything from Amazon Q’s brand and security features to supply chain and coding overview, the post said.
Neighorn told BI that sales training was “standard practice” for tech companies and any suggestion that it was unusual was “false.”
Garman seems to have high hopes for AI assistants like Amazon Q. At an internal conference in June, he said that software engineers could soon stop coding because of advances in these types of AI tools. Amazon is also working on a separate AI chatbot, internally codenamed Metis.
For now, Amazon CEO Andy Jassy may be Amazon Q’s biggest seller. Last month, he wrote on LinkedIn that Amazon’s internal use of Q has helped the company gain “increased efficiencies,” adding that it has saved the company “4,500 developer years of work” and about $260 million in “annualized efficiencies.”
“This has been a game changer for us, and not only do our Amazon teams plan to use this transformation capability more, but our Q team plans to add more transformations for developers to take advantage of,” Jassy wrote.
Do you work at Amazon? Do you have a tip?
Contact the journalist, Eugene Kim, via the encrypted messaging apps Signal or Telegram (+1-650-942-3061) or by email (ekim@businessinsider.com). Contact us using a non-work device. Visit Business Insider source guide for more tips on sharing information securely.
businessinsider
