How to build better conversational experiences with generative AI

foreign
hello and welcome my name is Chris
Overholt and I'm a developer Advocate
who works with conversational AI in
Google Cloud
I'm very excited to talk with you today
about how to boost your conversational
experiences with generative AI
we'll also get into the details of how
you can quickly build out an amazing
customer experience with something
called gen app builder so let's get
started
first we'll review the current
capabilities of chat Bots and how they
affect the customer experience today
then we'll get into specific aspects of
the next generation of conversational
experiences including informational
transactional and generative
capabilities
we'll start with a customer Journey that
most of us have probably been through
firsthand a frustrating experience with
a chat bot or a virtual phone agent
let's say that you want to buy tickets
for a concert in San Francisco on May
5th
the call starts with a greeting then the
virtual agent proceeds to ask one
question after the other about the
location the performer the date the
number of tickets and lots of
confirmations along the way
after this very long and painful
interrogation process it turns out there
are no tickets available for the date
that you actually wanted anyway
we can see that this interaction was
very mechanical and robotic and the
customer experience was similar to
filling out a long and tedious form one
field at a time
and only after you've completed the
entire form in the end you're
disappointed that we can't help you
anyway
this kind of interaction usually ends
with a customer repeatedly yelling into
their phone customer support customer
support customer support until they're
ultimately transferred to a human agent
to handle their issue
now instead let's walk through a more
natural and pleasant customer experience
in this case rather than following
prompts from the chat bot the customer
just speaks in plain natural language
and States all of the important details
up front just like they would when
talking to another person in real life
I'd like to buy two tickets for this
particular concert in San Francisco
then we move on to available dates and
payment
here we use natural language
understanding and machine learning
models on the back end and the chat bot
was able to deconstruct parameterize and
act much more quickly and efficiently
with the customer
now even when you have powerful natural
language understanding models on the
back end
implementing a virtual agent is still a
very complex process
it's very common for chatbot developers
to end up with virtual agents that have
hundreds or even thousands of components
since I need to represent all of the
different paths that a customer might
take along the way
this results in a very large and complex
conversation flow that's difficult to
implement and even more difficult to
update just like the conversation graph
you see on the right side
as a result developers often get lost in
the logic and the routing of the virtual
agents which takes away valuable time
and effort that could have gone into
designing a more pleasant customer
experience
and this is exactly where generative AI
comes in to help us better align our
chat Bots with conversation design
principles
if we follow fundamental conversation
design principles and our development
tool handles the implementation details
for us
then we're much more likely to end up
with a simple and intuitive user
experience which means happier customers
increased call containment rates and an
overall better experience for customers
for the rest of this session we'll focus
on three types of conversational
capabilities that you can combine to
create amazing customer experiences
first informational capabilities which
means that customers can ask more
open-ended questions with natural
language to quickly get the information
they need without chatbot developers
having to manually Define thousands of
questions and answers
second transactional capabilities this
means that chatbots can now handle a
wider range of tasks and actions
automatically rather than needing to
escalate to a human agent early in the
conversation
and third generative capabilities this
means combining the first two
capabilities to make the customer
experience hyper personalized for
example we could generate or rewrite
content in real time based on how the
question was phrased or even based on a
specific customer profile
now that we understand the specific
challenges and complexities involved
with implementing conversational
experiences let's walk through the
timeline of large language models and
generative AI capabilities so we can
understand how we got here and how we
can use these together to improve the
customer experience
we'll start by discussing what
generative AI is and what large language
models are
large language models are built on top
of an understanding of general knowledge
because they're trained on a very large
Corpus or a set of documents
because of this llms can handle much
more complex tasks like Auto completion
information retrieval sentiment analysis
and intent detection
and if you're wondering about the term
large in large language models it means
that these models have on the order of
billions of parameters
and generative AI is the application of
these kinds of models to produce content
such as text images video and code
in the case of chat Bots this generated
content can actually help customers
solve their problems Faster by pulling
from a larger realm of general
information
now if we look at the timeline in terms
of machine learning and AI developments
Google has always been at the Forefront
of AI you can see this both in our
research and in our products
over the last few years we've made
significant investments into building
and training large language models
back in 2017 there was a breakthrough
research paper on Transformers published
by members of the Google brain team
at that time Transformers represented a
huge leap in how well these models
performed in the real world
and ever since then Google's released
several other large language models like
the ones you see here on the timeline
alongside all of this research we've
also been productionizing and making
these Technologies usable for our own
customers in Google Cloud
we're very excited to continue to
translate these technical breakthroughs
into products that help billions of
people around the world
now you might be wondering where does
all of this actually fit into Google
Cloud
a couple of months ago we announced a
few new products related to generative
AI including Foundation models
Enterprise search and conversational AI
these products help developers build
apps that surface generative AI
capabilities in lots of different ways
for example Foundation models are
available in vertex Ai and they include
generative functionality for things like
text code and images
Enterprise search complements the
foundation models to provide up-to-date
and targeted information from both
internal data sources and public
websites
conversational AI which is the main
focus of our talk today enables
developers to steer customers towards
productive actions using informational
capabilities and can answer questions
like how much would my phone bill change
if I added international data roaming or
even transactional capabilities like
help me pay this month's phone bill
now we're going to focus on
conversational Ai and learn how
generative AI can enhance customer
experiences
specifically we're going to take a
deeper look into the capabilities
provided by generative AI app builder or
gen app builder for short
with Gen app builder we're going to look
at the informational transactional and
generative capabilities that we
described earlier
for now you can think of gen app builder
as a very fast and easy way for
developers to quickly create apps like
chat Bots virtual support assistants and
custom search engines with minimal
development overhead
and behind the scenes the conversational
capabilities in gen app builder are
powered by Google Cloud's natural
language models speech models and of
course large language models
we've been helping our users deliver
more natural chat and voice-based
experiences over the last several years
and we do this by using virtual agents
that support natural language
understanding multi-turn conversations
and conversational technologies that are
built with dialogflow in Google cloud
and the same deep learning technologies
that power the Google Assistant
now for the latest iteration of virtual
agents that we're talking about today
we're really just getting started with
llm powered conversational experiences
and generative Ai and it's going to
drastically change the way that we
develop virtual agents as well as how
customers interact with them
in this section we'll talk about a few
different ways that gen app builder can
help you build amazing conversational
experiences both in terms of faster
chatbot development and directly
improving customer experience
in the past if you needed to provide an
informational chat experience on your
website that could answer hundreds of
different questions you would actually
need to manually create a very large
amount of intents and responses in your
chatbot
then you would need to specify
additional variants that represent
different ways that a customer might ask
their question
but with Gen app builder you can skip
over all of that tedious step-by-step
process and easily create an FAQ based
informational bot that uses existing
content from your support knowledge
bases or even your Internal
Documentation
on the back end Genet Builder uses
Advanced natural language models to
parse and tokenize your website and FAQ
content organize it into question and
answer Pairs and then route users to the
most appropriate answers at the right
time
if you use this approach you can quickly
set up a conversational app that can be
embedded on your website or mobile app
and it's instantly ready to help your
customers find the right answers
another capability that we'll discuss is
called generative fallback responses
which also focuses on customer facing
conversational functionality
in the past if a user said something
that was not represented in the logical
design of a chat bot the virtual agent
would fall back to a default intent and
respond with some variation of I'm sorry
I didn't get that can you rephrase
and as we saw in the example of a
frustrating customer experience this is
a very common point of irritation for
customers when it happens over and over
during a call or a chat session
instead of trying to design for hundreds
of potential edge cases and potentially
over engineering our chat bot we can
instead use generative Ai and large
language models to create responses for
users in real time
for example
we might have a customer who has some
questions about travel policies before
they actually purchase a flight
or we might have another customer who
wants to know about Regional Healthcare
information in a particular country
before they travel
instead of leaving these customers in
the dark with inactionable responses we
can actually use generative responses in
our conversation to help us get
customers back on the happy path in the
conversation flow
foreign
now we'll move on to more development
focused conversation tasks
the next capability we'll discuss helps
developers build conversational
experiences faster using natural
language prompts
earlier we talked about how developers
can manually create intents routes pages
and flows within chat Bots they can do
this either using the visual flow
designer or by using client libraries in
Python Java node.js and other languages
but if we go one step further what if we
could just describe the task that we
want the chatbot to perform and then our
development tool handled all of the
details and gave us back a fully
implemented conversation flow again
that's where gen app builder comes in
as you can see here we don't have to
deal with flows intents and other
low-level objects but instead we
describe what our virtual agent should
do to help the user including the task
definition the end goal parameters and
even API endpoints and validation checks
that should happen along the way
gen app builder then uses a large
language model on the back end to parse
all of these instructions and quickly
build out a virtual agent that's ready
for us to use
you can think of this capability as a
way for chatbot developers to focus more
on the conversational design and less on
the implementation details all thanks to
the power of llms and generative AI
the final capability that we'll talk
about also focuses on speeding up the
development time of virtual agents but
this time we're going to talk about
handling complex graphs and state
transitions more efficiently in
conversation flows
essentially you describe the
conversation flow at a very high level
and then an llm based intelligence model
generates and auto completes a detailed
conversation graph and helps you quickly
build out a more comprehensive customer
experience
of course you can still customize every
aspect of the conversation flow and dig
deeper into specific points of the
customer Journey when you need to
the traditional conversational
development console is still there
underneath gen app builder if you need
more control or if you want to preserve
backward compatibility with your
existing virtual agents
looking back at this session we covered
a lot of ground in terms of generative
AI capabilities and new approaches for
llm-powered chatbots
but what does this actually mean for the
future of customer experiences
it means new generative conversational
capabilities easier chatbot development
and The Best of Both Worlds when it
comes to combining domain-specific
information with general knowledge
I'll wrap up our session today with a
question that we get asked about a lot
in recent times
in a world where there are large
language models and generative AI why do
we even need chat Bots at all
why not just drop a large language model
on my company website
and move on to other interesting
problems well
the answer has a lot to do with the
scope of a large language model versus
the scope of a virtual agent so if you
remember from earlier a large language
model is trained on a massive Corpus
which results in a model that acts like
a generalist across lots of different
topics and it's really designed to
handle open-ended conversational
dialogues or content generation
on the other hand a virtual agent is a
curated custom built conversational bot
that can guide customers on an optimal
path to a specific solution
so if you take one thing away from our
discussion today just know you don't
have to choose between generative AI or
chat Bots with Gen app builder you
actually have access to more options to
provide a great customer experience that
captures The Best of Both Worlds Rich AI
generated content along with
goal-oriented conversation steering
now I want to leave you with a few
different resources you can visit for
more information
first up if you want to learn more about
generative Ai and Google Cloud including
everything from Foundation models to
Enterprise grade generative AI Solutions
you can visit the first link
if you actually want to try out and give
feedback on emerging AI Technologies
like Genet Builder you can visit the
second link to join the waitlist for our
trusted tester program
and finally as you're exploring and
trying things out yourself we'd love to
hear what your experience is like and
how we can help you build an amazing
experience for your own customers you
can get in touch by using that third
link
as you can tell we are very excited
about the new conversational
capabilities powered by generative Ai
and the positive impact they'll have on
improving customer experiences
everywhere
thanks so much for your time and for
attending the session I hope you learned
something that will be useful in your
own journey to build amazing customer
experiences and I hope that you enjoy
the rest of your time at Google i o
thank you
[Music]
thank you