Google Developers Blog

 

18th January 2018 |

Google Play Games Services C++ SDK 3.0 Released
Posted by Clayton Wilkinson, Developer Relations

We're pleased to announce the availability of the Google Play Games Services C++ SDK version 3.0. The highlights of this release are:

  • Requires Android NDK r14 or greater.
  • Compiled using the clang toolchain. The use of clang with projects using this SDK is strongly recommended in order to avoid unexpected behavior.
  • The armeabi ABI has been removed. You should use armeabi-v7a.
  • Bug fixes for the Nearby API
  • Refinements in the Snapshots API.

More details can be found in the release notes on the downloads page.

The SDK can be downloaded from: https://developers.google.com/games/services/downloads/sdks

Samples using this SDK can be downloaded from GitHub: https://github.com/playgameservices/cpp-android-basic-samples

Thanks and happy coding!

 

10th January 2018 |

Real-world data in PageSpeed Insights
Posted by Mushan Yang and Xiangyu Luo, Software Engineers

PageSpeed Insights provides information about how well a page adheres to a set of best practices. In the past, these recommendations were presented without the context of how fast the page performed in the real world, which made it hard to understand when it was appropriate to apply these optimizations. Today, we're announcing that PageSpeed Insights will use data from the Chrome User Experience Report to make better recommendations for developers and the optimization score has been tuned to be more aligned with the real-world data.

The PSI report now has several different elements:

  • The Speed score categorizes a page as being Fast, Average, or Slow. This is determined by looking at the median value of two metrics: First Contentful Paint (FCP) and DOM Content Loaded (DCL). If both metrics are in the top one-third of their category, the page is considered fast.
  • The Optimization score categorizes a page as being Good, Medium, or Low by estimating its performance headroom. The calculation assumes that a developer wants to keep the same appearance and functionality of the page.
  • The Page Load Distributions section presents how this page's FCP and DCL events are distributed in the data set. These events are categorized as Fast (top third), Average (middle third), and Slow (bottom third) by comparing to all events in the Chrome User Experience Report.
  • The Page Stats section describes the round trips required to load the page's render-blocking resources, the total bytes used by the page, and how it compares to the median number of round trips and bytes used in the dataset. It can indicate if the page might be faster if the developer modifies the appearance and functionality of the page.
  • Optimization Suggestions is a list of best practices that could be applied to this page. If the page is fast, these suggestions are hidden by default, as the page is already in the top third of all pages in the data set.

For more details on these changes, see About PageSpeed Insights. As always, if you have any questions or feedback, please visit our forumsand please remember to include the URL that is being evaluated.

 

9th January 2018 |

Actions on Google: new directory, device availability and smart home controls
Posted by Brad Abrams, Product Manager

With the Google Assistant and Actions on Google, we're excited for 2018 and look forward to continuing the developer momentum you've helped us build. To start the year off right, we're at the Consumer Electronics Show in Las Vegas showcasing the Assistant at home, on the go and in the car—and all the ways it can help in each of those places. You can learn more here. For developers like you, we're building upon those same areas to extend the ways you can reach users in those places, too.

Helping users get more done, together

Today we're introducing a new web directory and an updated directoryexperience with the Assistant on phones. These directories give users even more visibility into everything your app can help them do. They also make it even easier for users to share links to your apps. And together with your help, we're adding Actions all the time including those that are coming soon from SpotHero and Starbucks.

Even better, when you publish your first app, you'll become eligible for our developer community program, that supports you with up to $200 in monthly Google Cloud credit and an Assistant t-shirt - with the perks and opportunities growing the more you do, including earning a Google Home.

At home, on the go and in the car

With the Assistant, your apps are available across many devices and this year, we're making them even more available with new integrations at home, on the go and in the car.

For the home, we announced that smart displays with the Assistant built in are coming later this year. Smart displays come with the added benefit of a touch screen, they can help provide a visual experience for users.

Beyond smart displays, we also have the Assistant coming to new speakers and TVs with the Assistant built in, as well as new headphones that are optimized for the Assistant.

Finally, starting later this week, we're bringing the Assistant to Android Auto, allowing users to project Android Auto, and with it the Assistant, onto the screen in their compatible car.

The best part is that compatible apps will be available to users on all these devices without additional work. With that said, to ensure the best user experience, here are a few tips:

  • Smart displays — use high resolution imagery as users will be interacting with larger images than those sized for phones.
  • Android Auto — since this experience is in a car and only voice-only apps will be available, keep voice interaction and sounds simple and not too jarring or distracting.

More control of your home with smart home control

In addition to the enhanced home experience with built in devices, we're also updating our home control experience, making it easier than ever to build for smart homes. The Google Assistant already works with more than 1,500 smart devices from 200+ brands, but this is still just the start for the number of devices we anticipate will be built for the smart home.

We first launched the smart home Actionsat I/O this year and we started with support for things like lights, plugs and thermostats. Now, we're excited to announce we've added direct support for a number of new device types, including: cameras, dishwashers, dryers, vacuumsand washers. This means that users can control all kinds of appliances in their home just by asking the Google Assistant. And in order to support these new integrations, we're also expanding the supported device traitsto include: camerastream, dock, modes, runcycle, scene, start/stopand toggles. With all these new devices, it is a good thing we have made it even easier to build smart home Actions with a streamlined development flow and insightful analytics to help you improve your smart home Action. Ready to begin? Start here!

And that's our news for now. Thanks for everything you do to make the Assistant more helpful, fun and interactive! It's been an exciting year to see the platform expand to new languages and devices and to see what you've all created. We can't wait to see what you build and the new ways users are able to get things done as a result. Here's to a great year!

 

2nd January 2018 |

Harness your data with Data Studio Community Connectors
Posted by Minhaz Kazi, Developer Advocate

Google Data Studio lets users build live, interactive dashboards with beautiful data visualizations, for free. Users can fetch their data from a variety of sources and create unlimited reports in Data Studio, with full editing and sharing capabilities.

Community Connectors is a new feature for Data Studio that lets you use Apps Script to build connectors to any internet accessible data source. You can share Community Connectors with other people so they can access their own data from within Data Studio.

For example, if you are providing a web-based service to your customers, you can create a Community Connector with a template dashboard to fetch data from your API. In just 3 to 4 clicks, your customers can log into your web app, authenticate with Data Studio, and see their individualized data displayed in a beautiful interactive dashboard.

Here's an example Data Studio dashboard that uses a Community Connector to show live data using the Stack Overflow API:

Try out this Stack Overflow Community Connector yourself or view the code.

Why build Community Connectors

  • Leverage Data Studio as a reporting platform for your customers.
    Provide significant value to your customers by providing them with a ready-made reporting platform. With a minimal development investment, you can rely on Data Studio as a free and powerful dashboarding and analysis solution for your customers.

  • Reach a larger audience and also monetize your connector.
    Publish and promote your Community Connector in the Data Studio Community Connector gallery that is visible to all Data Studio users. Published connectors are also directly accessible from the public Community Connector Gallery. There are also multiple approaches if you want to monetize your connector.

  • Develop customized enterprise solutions for your business.
    Fetch your business data from a variety of sources (e.g. BigQuery, CloudSQL, web API etc.) and create a customized solution specifically for your business. By providing templates with your connectors, you can significantly cut down dashboard building time.

  • Benefit from Apps Script features and use your existing code.
    Since Community Connectors are developed using Google Apps Script, you can benefit from features such as caching, storage, translation, authentication etc. If you already have a Google Sheets connector, it is easy to reuse that same code for a Community Connector.

  • Did we mention it's free?
    Data Studio is completely free to use. And there is no cost for developing or publishing Community Connectors.

How to build Community Connectors

The Get Started Guide can help you to build your first Community Connector. Since Apps Script is a subset of Javascript, you can easily build a connector even if you have not worked with Apps Script before.

You can also jump ahead and view specific steps of the typical development life cycle of a Community Connector:

  • Build: Write your connector code in Apps Script.
  • Use and test: Try out your connector in Data Studio.
  • Deploy: Create deployments for other users.
  • Share: Share your connector.

Publish your connector

You can keep your connector private or share them with other users. You also have the option to publish your connector. Publishing will feature your connector both in Data Studio as well as in the public Community Connector gallery. This enables you to reach all Data Studio users and showcase your service. Furthermore, we encourage you to submit your connector to our Open Source repo so that the community can benefit from it.

What next

If you have any interesting connector stories, ideas, or if you'd like to share some amazing reports you've created using Community Connectors, give us a shout or send us your story at community-connector-feedback@google.com.

 

21st December 2017 |

The Google Assistant SDK - new languages, API updates, and Device Actions
Posted by Glen Shires, Google Assistant SDK Tech Lead

The Google Assistant SDK lets developers like you embed the Google Assistant into any device with a microphone and speaker. Since we first introduced the SDK, you've created innovative projects and delightful applications with Voice Kits. Your fun side projects and practical applications have captivated our imagination, and we'll continue working with companies—big and small—to develop and launch new products to extend the availability of the Google Assistant.

To help you take your products to the next level, today we're happy to introduce several new features to the Google Assistant SDK.

Additional languages and locales

Supporting users globally is important for the Google Assistant and as of the latest release you can now programmatically configure the API, or configure your device within the Assistant app, to use any of the following languages/locales: English (Australia, Canada, UK, US), French (Canada, France), German, and Japanese.

Customizable device settings

Many aspects of the Google Assistant can be customizedby end-users in the Settings screen within the Assistant on their phone. SDK-based devices are not only discoverable within this experience, but they also support the same level of customization, including changing the device's language, location, nickname, and enabling personalized results -- for example, "Ok Google, what's on my calendar?"

In terms of location, SDK-based devices can now be configured as a street address in the Google Assistant on your phone, or as a latitude and longitude via the API. With this ability, SDK-based devices can return more location-specific answers to queries such as "Ok Google, where's the nearest coffee shop?" or "Ok Google, what's today's weather?"

Text-based queries and responses

Voice-in and voice-out was a natural first step for the Google Assistant SDK, but we have heard from many developers that other input and output mechanisms are needed. Today we're happy to announce that the Google Assistant SDK now supports text-based queries and responses. Both of these updates build upon the already-supported voice query and voice response API.

Device Actions

When we first launched the Google Assistant SDK one of the most prominent questions we received was "how can I ask the Assistant to control my device?" With the latest SDK, you can utilize the new Device Action functionality to build Actions directly into your Assistant-enabled SDK devices.

When you register a device you can now specify what traits the device itself supports – on/off or temperature setting, for example. When users then ask the device, "Ok Google, set the temperature to 78 degrees," the Google Assistant will turn such queries into structured intents via cloud-based automated speech recognition (ASR) and natural language understanding (NLU). All you need to provide is the client-side code for actually fulfilling the Device Action itself – no other code is needed. The SDK supports a set of device traits that are supported by Smart Home.

Device management

To help get you up and running with Device Actions, we are launching a new management APIto help you register and manage your SDK devices. With this API you are able to easily register, unregister, and see all devices that you have registered. We're also introducing a device model which represents a set of devices with the same type and traits.

Get started with all this new functionality, by checking out the documentation and samples.

If you're interested in building a commercial product with the Google Assistant, we encourage you to reach out and contact us.

As always, there are great conversations happening within StackOverflow, as well as the Assistant SDK and hackster.iocommunities. We encourage everyone to take part!

 

19th December 2017 |

Creating Custom Estimators in TensorFlow
Posted by the TensorFlow Team

Welcome to Part 3 of a blog series that introduces TensorFlow Datasets and Estimators. Part 1 focused on pre-made Estimators, while Part 2 discussed feature columns. Here in Part 3, you'll learn how to create your own custom Estimators. In particular, we're going to demonstrate how to create a custom Estimator that mimics DNNClassifier's behavior when solving the Iris problem.

If you are feeling impatient, feel free to compare and contrast the following full programs:

  • Source code for Iris implemented with the pre-made DNNClassifier Estimator here.
  • Source code for Iris implemented with the custom Estimator here.

Pre-made vs. custom

As Figure 1 shows, pre-made Estimators are subclasses of the tf.estimator.Estimatorbase class, while custom Estimators are an instantiation of tf.estimator.Estimator:

Figure 1. Pre-made and custom Estimators are all Estimators.

Pre-made Estimators are fully-baked. Sometimes though, you need more control over an Estimator's behavior. That's where custom Estimators come in.

You can create a custom Estimator to do just about anything. If you want hidden layers connected in some unusual fashion, write a custom Estimator. If you want to calculate a unique metric for your model, write a custom Estimator. Basically, if you want an Estimator optimized for your specific problem, write a custom Estimator.

A model function (model_fn) implements your model. The only difference between working with pre-made Estimators and custom Estimators is:

  • With pre-made Estimators, someone already wrote the model function for you.
  • With custom Estimators, you must write the model function.

Your model function could implement a wide range of algorithms, defining all sorts of hidden layers and metrics. Like input functions, all model functions must accept a standard group of input parameters and return a standard group of output values. Just as input functions can leverage the Dataset API, model functions can leverage the Layers API and the Metrics API.

Iris as a pre-made Estimator: A quick refresher

Before demonstrating how to implement Iris as a custom Estimator, we wanted to remind you how we implemented Iris as a pre-made Estimator in Part 1 of this series. In that Part, we created a fully connected, deepneural network for the Iris dataset simply by instantiating a pre-made Estimator as follows:

# Instantiate a deep neural network classifier.
classifier = tf.estimator.DNNClassifier(
feature_columns=feature_columns, # The input features to our model.
hidden_units=[10, 10], # Two layers, each with 10 neurons.
n_classes=3, # The number of output classes (three Iris species).
model_dir=PATH) # Pathname of directory where checkpoints, etc. are stored.

The preceding code creates a deep neural network with the following characteristics:

  • A list of feature columns. (The definitions of the feature columns are not shown in the preceding snippet.) For Iris, the feature columns are numeric representations of four input features.
  • Two fully connected layers, each having 10 neurons. A fully connected layer (also called a dense layer) is connected to every neuron in the subsequent layer.
  • An output layer consisting of a three-element list. The elements of that list are all floating-point values; the sum of those values must be 1.0 (this is a probability distribution).
  • A directory (PATH) in which the trained model and various checkpoints will be stored.

Figure 2 illustrates the input layer, hidden layers, and output layer of the Iris model. For reasons pertaining to clarity, we've only drawn 4 of the nodes in each hidden layer.

Figure 2. Our implementation of Iris contains four features, two hidden layers, and a logits output layer.

Let's see how to solve the same Iris problem with a custom Estimator.

Input function

One of the biggest advantages of the Estimator framework is that you can experiment with different algorithms without changing your data pipeline. We will therefore reuse much of the input function from Part 1:

def my_input_fn(file_path, repeat_count=1, shuffle_count=1):
def decode_csv(line):
parsed_line = tf.decode_csv(line, [[0.], [0.], [0.], [0.], [0]])
label = parsed_line[-1] # Last element is the label
del parsed_line[-1] # Delete last element
features = parsed_line # Everything but last elements are the features
d = dict(zip(feature_names, features)), label
return d

dataset = (tf.data.TextLineDataset(file_path) # Read text file
.skip(1) # Skip header row
.map(decode_csv, num_parallel_calls=4) # Decode each line
.cache() # Warning: Caches entire dataset, can cause out of memory
.shuffle(shuffle_count) # Randomize elems (1 == no operation)
.repeat(repeat_count) # Repeats dataset this # times
.batch(32)
.prefetch(1) # Make sure you always have 1 batch ready to serve
)
iterator = dataset.make_one_shot_iterator()
batch_features, batch_labels = iterator.get_next()
return batch_features, batch_labels

Notice that the input function returns the following two values:

  • batch_features, which is a dictionary. The dictionary's keys are the names of the features, and the dictionary's values are the feature's values.
  • batch_labels, which is a list of the label's values for a batch.

Refer to Part 1 for full details on input functions.

Create feature columns

As detailed in Part 2 of our series, you must define your model's feature columns to specify the representation of each feature. Whether working with pre-made Estimators or custom Estimators, you define feature columns in the same fashion. For example, the following code creates feature columns representing the four features (all numerical) in the Iris dataset:

feature_columns = [
tf.feature_column.numeric_column(feature_names[0]),
tf.feature_column.numeric_column(feature_names[1]),
tf.feature_column.numeric_column(feature_names[2]),
tf.feature_column.numeric_column(feature_names[3])
]

Write a model function

We are now ready to write the model_fn for our custom Estimator. Let's start with the function declaration:

def my_model_fn(
features, # This is batch_features from input_fn
labels, # This is batch_labels from input_fn
mode): # Instance of tf.estimator.ModeKeys, see below

The first two arguments are the features and labels returned from the input function; that is, features and labels are the handles to the data your model will use. The mode argument indicates whether the caller is requesting training, predicting, or evaluating.

To implement a typical model function, you must do the following:

  • Define the model's layers.
  • Specify the model's behavior in three the different modes.

Define the model's layers

If your custom Estimator generates a deep neural network, you must define the following three layers:

  • an input layer
  • one or more hidden layers
  • an output layer

Use the Layers API (tf.layers) to define hidden and output layers.

If your custom Estimator generates a linear model, then you only have to generate a single layer, which we'll describe in the next section.

Define the input layer

Call tf.feature_column.input_layerto define the input layer for a deep neural network. For example:

# Create the layer of input
input_layer = tf.feature_column.input_layer(features, feature_columns)

The preceding line creates our input layer, reading our featuresthrough the input function and filtering them through the feature_columns defined earlier. See Part 2 for details on various ways to represent data through feature columns.

To create the input layer for a linear model, call tf.feature_column.linear_modelinstead of tf.feature_column.input_layer. Since a linear model has no hidden layers, the returned value from tf.feature_column.linear_model serves as both the input layer and output layer. In other words, the returned value from this function isthe prediction.

Establish Hidden Layers

If you are creating a deep neural network, you must define one or more hidden layers. The Layers API provides a rich set of functions to define all types of hidden layers, including convolutional, pooling, and dropout layers. For Iris, we're simply going to call tf.layers.Densetwice to create two dense hidden layers, each with 10 neurons. By "dense," we mean that each neuron in the first hidden layer is connected to each neuron in the second hidden layer. Here's the relevant code:

# Definition of hidden layer: h1
# (Dense returns a Callable so we can provide input_layer as argument to it)
h1 = tf.layers.Dense(10, activation=tf.nn.relu)(input_layer)

# Definition of hidden layer: h2
# (Dense returns a Callable so we can provide h1 as argument to it)
h2 = tf.layers.Dense(10, activation=tf.nn.relu)(h1)

The inputs parameter to tf.layers.Dense identifies the preceding layer. The layer preceding h1 is the input layer.

Figure 3. The input layer feeds into hidden layer 1.

The preceding layer to h2 is h1. So, the string of layers now looks like this:

Figure 4. Hidden layer 1 feeds into hidden layer 2.

The first argument to tf.layers.Densedefines the number of its output neurons—10 in this case.

The activation parameter defines the activation function—Relu in this case.

Note that tf.layers.Denseprovides many additional capabilities, including the ability to set a multitude of regularization parameters. For the sake of simplicity, though, we're going to simply accept the default values of the other parameters. Also, when looking at tf.layersyou may encounter lower-case versions (e.g. tf.layers.dense). As a general rule, you should use the class versions which start with a capital letter (tf.layers.Dense).

Output Layer

We'll define the output layer by calling tf.layers.Dense yet again:

# Output 'logits' layer is three numbers = probability distribution
# (Dense returns a Callable so we can provide h2 as argument to it)
logits = tf.layers.Dense(3)(h2)

Notice that the output layer receives its input from h2. Therefore, the full set of layers is now connected as follows:

Figure 5. Hidden layer 2 feeds into the output layer.

When defining an output layer, the units parameter specifies the number of possible output values. So, by setting units to 3, the tf.layers.Densefunction establishes a three-element logits vector. Each cell of the logits vector contains the probability of the Iris being Setosa, Versicolor, or Virginica, respectively.

Since the output layer is a final layer, the call to tf.layers.Denseomits the optional activation parameter.

Implement training, evaluation, and prediction

The final step in creating a model function is to write branching code that implements prediction, evaluation, and training.

The model function gets invoked whenever someone calls the Estimator's train, evaluate, or predict methods. Recall that the signature for the model function looks like this:

def my_model_fn(
features, # This is batch_features from input_fn
labels, # This is batch_labels from input_fn
mode): # Instance of tf.estimator.ModeKeys, see below

Focus on that third argument, mode. As the following table shows, when someone calls train, evaluate, or predict, the Estimator framework invokes your model function with the mode parameter set as follows:

Table 2. Values of mode.

Caller invokes custom Estimator method... Estimator framework calls your model function with the mode parameter set to...
train() ModeKeys.TRAIN
evaluate() ModeKeys.EVAL
predict() ModeKeys.PREDICT

For example, suppose you instantiate a custom Estimator to generate an object named classifier. Then, you might make the following call (never mind the parameters to my_input_fn at this time):

classifier.train(
input_fn=lambda: my_input_fn(FILE_TRAIN, repeat_count=500, shuffle_count=256))

The Estimator framework then calls your model function with mode set to ModeKeys.TRAIN.

Your model function must provide code to handle all three of the mode values. For each mode value, your code must return an instance of tf.estimator.EstimatorSpec, which contains the information the caller requires. Let's examine each mode.

PREDICT

When model_fn is called with mode == ModeKeys.PREDICT, the model function must return a tf.estimator.EstimatorSpeccontaining the following information:

  • the mode, which is tf.estimator.ModeKeys.PREDICT
  • the prediction

The model must have been trained prior to making a prediction. The trained model is stored on disk in the directory established when you instantiated the Estimator.

For our case, the code to generate the prediction looks as follows:

# class_ids will be the model prediction for the class (Iris flower type)
# The output node with the highest value is our prediction
predictions = { 'class_ids': tf.argmax(input=logits, axis=1) }

# Return our prediction
if mode == tf.estimator.ModeKeys.PREDICT:
return tf.estimator.EstimatorSpec(mode, predictions=predictions)

The block is surprisingly brief--the lines of code are simply the bucket at the end of a long hose that catches the falling predictions. After all, the Estimator has already done all the heavy lifting to make a prediction:

  1. The input function provides the model function with data (feature values) to infer from.
  2. The model function transforms those feature values into feature columns.
  3. The model function runs those feature columns through the previously-trained model.

The output layer is a logits vector that contains the value of each of the three Iris species being the input flower. The tf.argmaxmethod selects the Iris species in that logits vector with the highest value.

Notice that the highest value is assigned to a dictionary key named class_ids. We return that dictionary through the predictions parameter of tf.estimator.EstimatorSpec. The caller can then retrieve the prediction by examining the dictionary passed back to the Estimator's predict method.

EVAL

When model_fn is called with mode == ModeKeys.EVAL, the model function must evaluate the model, returning loss and possibly one or more metrics.

We can calculate loss by calling tf.losses.sparse_softmax_cross_entropy. Here's the complete code:

# To calculate the loss, we need to convert our labels
# Our input labels have shape: [batch_size, 1]
labels = tf.squeeze(labels, 1) # Convert to shape [batch_size]
loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)

Now let's turn our attention to metrics. Although returning metrics is optional, most custom Estimators return at least one metric. TensorFlow provides a Metrics API (tf.metrics) to calculate different kinds of metrics. For brevity's sake, we'll only return accuracy. The tf.metrics.accuracycompares our predictions against the "true labels", that is, against the labels provided by the input function. The tf.metrics.accuracyfunction requires the labels and predictions to have the same shape (which we did earlier). Here's the call to tf.metrics.accuracy:

# Calculate the accuracy between the true labels, and our predictions
accuracy = tf.metrics.accuracy(labels, predictions['class_ids'])

When the model is called with mode == ModeKeys.EVAL, the model function returns a tf.estimator.EstimatorSpec containing the following information:

  • the mode, which is tf.estimator.ModeKeys.EVAL
  • the model's loss
  • typically, one or more metrics encased in a dictionary.

So, we'll create a dictionary containing our sole metric (my_accuracy). If we had calculated other metrics, we would have added them as additional key/value pairs to that same dictionary. Then, we'll pass that dictionary in the eval_metric_ops argument of tf.estimator.EstimatorSpec. Here's the block:

# Return our loss (which is used to evaluate our model)
# Set the TensorBoard scalar my_accurace to the accuracy
# Obs: This function only sets value during mode == ModeKeys.EVAL
# To set values during training, see tf.summary.scalar
if mode == tf.estimator.ModeKeys.EVAL:
return tf.estimator.EstimatorSpec(
mode,
loss=loss,
eval_metric_ops={'my_accuracy': accuracy})

TRAIN

When model_fn is called with mode == ModeKeys.TRAIN, the model function must train the model.

We must first instantiate an optimizer object. We picked Adagrad (tf.train.AdagradOptimizer) in the following code block only because we're mimicking the DNNClassifier, which also uses Adagrad. The tf.trainpackage provides many other optimizers—feel free to experiment with them.

Next, we train the model by establishing an objective on the optimizer, which is simply to minimize its loss. To establish that objective, we call the minimizemethod.

In the code below, the optional global_step argument specifies the variable that TensorFlow uses to count the number of batches that have been processed. Setting global_step to tf.train.get_global_stepwill work beautifully. Also, we are calling tf.summary.scalarto report my_accuracy to TensorBoard during training. For both of these notes, please see the section on TensorBoard below for further explanation.

optimizer = tf.train.AdagradOptimizer(0.05)
train_op = optimizer.minimize(
loss,
global_step=tf.train.get_global_step())

# Set the TensorBoard scalar my_accuracy to the accuracy
tf.summary.scalar('my_accuracy', accuracy[1])

When the model is called with mode == ModeKeys.TRAIN, the model function must return a tf.estimator.EstimatorSpeccontaining the following information:

  • the mode, which is tf.estimator.ModeKeys.TRAIN
  • the loss
  • the result of the training op

Here's the code:

# Return training operations: loss and train_op
return tf.estimator.EstimatorSpec(
mode,
loss=loss,
train_op=train_op)

Our model function is now complete!

The custom Estimator

After creating your new custom Estimator, you'll want to take it for a ride. Start by

instantiating the custom Estimator through the Estimatorbase class as follows:

classifier = tf.estimator.Estimator(
model_fn=my_model_fn,
model_dir=PATH) # Path to where checkpoints etc are stored

The rest of the code to train, evaluate, and predict using our estimator is the same as for the pre-made DNNClassifierdescribed in Part 1. For example, the following line triggers training the model:

classifier.train(
input_fn=lambda: my_input_fn(FILE_TRAIN, repeat_count=500, shuffle_count=256))

TensorBoard

As in Part 1, we can view some training results in TensorBoard. To see this reporting, start TensorBoard from your command-line as follows:

# Replace PATH with the actual path passed as model_dir
tensorboard --logdir=PATH

Then browse to the following URL:

localhost:6006 

All the pre-made Estimators automatically log a lot of information to TensorBoard. With custom Estimators, however, TensorBoard only provides one default log (a graph of loss) plus the information we explicitly tell TensorBoard to log. Therefore, TensorBoard generates the following from our custom Estimator:

Figure 6. TensorBoard displays three graphs.

In brief, here's what the three graphs tell you:

  • global_step/sec: A performance indicator, showing how many batches (gradient updates) we processed per second (y-axis) at a particular batch (x-axis). In order to see this report, you need to provide a global_step (as we did with tf.train.get_global_step()). You also need to run training for a sufficiently long time, which we do by asking the Estimator train for 500 epochs when we call its train method:
    • loss: The loss reported. The actual loss value (y-axis) doesn't mean much. The shape of the graph is what's important.
  • my_accuracy: The accuracy recorded when we invoked both of the following:
  • eval_metric_ops={'my_accuracy': accuracy}), during EVAL (when returning our EstimatorSpec)
  • tf.summary.scalar('my_accuracy', accuracy[1]), during TRAIN

Note the following in the my_accuracy and loss graphs:

  • The orange line represents TRAIN.
  • The blue dot represents EVAL.

During TRAIN, orange values are recorded continuously as batches are processed, which is why it becomes a graph spanning x-axis range. By contrast, EVAL produces only a single value from processing all the evaluation steps.

As suggested in Figure 7, you may see and also selectively disable/enable the reporting for training and evaluation the left side. (Figure 7 shows that we kept reporting on for both:)

Figure 7. Enable or disable reporting.

In order to see the orange graph, you must specify a global step. This, in combination with getting global_steps/sec reported, makes it a best practice to always register a global step by passing tf.train.get_global_step()as an argument to the optimizer.minimize call.

Summary

Although pre-made Estimators can be an effective way to quickly create new models, you will often need the additional flexibility that custom Estimators provide. Fortunately, pre-made and custom Estimators follow the same programming model. The only practical difference is that you must write a model function for custom Estimators. Everything else is the same!

For more details, be sure to check out:

Until next time - Happy TensorFlow coding!

 

9th January 2018 |

Introducing the winners of the Actions on Google Developer Challenge
Posted by Akansha Sharma, Product Marketing Manager

This past year we worked hard to make the Google Assistant better for users and developers like you, but we also wanted to find new ways to reward you for doing what you love – building great apps for the Google Assistant.

So at I/O 2017, we announced our first Actions on Google Developer Challenge encouraging you to build helpful, entertaining apps for the Assistant. Today, we're announcing the competition's winners, chosen from thousands of entries.

  • First Place: 100 Years Ago
    • What it's all about: Travel back in time 100 years and listen to an interactive radio show. Catch up on breaking news and hit songs circa 1917.
  • Second Place: Credit Card Helper
    • What it's all about: Credit Card Helper analyzes card features, customer reviews, reads the fine print and looks up Consumer Reports to help you find the best credit card offers.
  • Third Place: My Adventure Book
    • What it's all about: Create your own adventure with Story Time. Listen to the story and direct the outcome with your choices.

In addition to the top three prize winners, we also selected winners among various categories including "best app by students," "best parenting app," "best life hack" and more. You can read up on all of the winners' apps here. Congratulations to our winners and to all those who submitted an app as part of the contest – we can't wait for users to check them out!

Happy holidays and happy New Year. We can't wait to see what the next year has in store.

Be sure to follow us on Twitter and check out the Google Assistant developer community program to stay in the know for 2018!

Correction: [January 4, 2018] Two previously announced winners were found ineligible according to the competition's terms. Updated winners availablehere.

 

7th December 2017 |

Announcing Core ML support in TensorFlow Lite
Posted by The TensorFlow Team

On November 14th, we announcedthe developer preview of TensorFlow Lite, TensorFlow's lightweight solution for mobile and embedded devices.

Today, in collaboration with Apple, we are happy to announce support for Core ML! With this announcement, iOS developers can leverage the strengths of Core ML for deploying TensorFlow models. In addition, TensorFlow Lite will continue to support cross-platform deployment, including iOS, through the TensorFlow Lite format (.tflite) as described in the original announcement.

Support for Core ML is provided through a tool that takes a TensorFlow model and converts it to the Core ML Model Format (.mlmodel).

For more information, check out the TensorFlow Lite documentation pages, and the Core ML converter. The pypi pip installable package is available here: https://pypi.python.org/pypi/tfcoreml/0.1.0.

Stay tuned for more updates.

Happy TensorFlow Lite coding!

 

4th December 2017 |

Getting Started with the Poly API
Posted by Bruno Oliveira, Software Engineer

As developers, we all know that having the right assets is crucial to the success of a 3D application, especially with AR and VR apps. Since we launched Poly a few weeks ago, many developers have been downloading and using Poly models in their apps and games. To make this process easier and more powerful, today we launched the Poly API, which allows applications to dynamically search and download 3D assets at both edit and run time.

The API is REST-based, so it's inherently cross-platform. To help you make the API calls and convert the results into objects that you can display in your app, we provide several toolkits and samples for some common game engines and platforms. Even if your engine or platform isn't included in this list, remember that the API is based on HTTP, which means you can call it from virtually any device that's connected to the Internet.

Here are some of the things the API allows you to do:

  • List assets, with many possible filters:
    • keyword
    • category ("Animals", "Technology", "Transportation", etc.)
    • asset type (Blocks, Tilt Brush, etc)
    • complexity (low, medium, high complexity)
    • curated (only curated assets or all assets)
  • Get a particular asset by ID
  • Get the user's own assets
  • Get the user's liked assets
  • Download assets. Formats vary by asset type (OBJ, GLTF1, GLTF2).
  • Download material files and textures for assets.
  • Get asset metadata (author, title, description, license, creation time, etc)
  • Fetch thumbnails for assets

Poly Toolkit for Unity Developers

If you are using Unity, we offer Poly Toolkit for Unity, a plugin that includes all the necessary functionality to automatically wrap the API calls and download and convert assets, exposing it through a simple C# API. For example, you can fetch and import an asset into your scene at runtime with a single line of code:

PolyApi.GetAsset(ASSET_ID,
result => { PolyApi.Import(result.Value, PolyImportOptions.Default()); });

Poly Toolkit optionally also handles authentication for you, so that you can list the signed in user's own private assets, or the assets that the user has liked on the Poly website.

In addition, Poly Toolkit for Unity also comes with an editor window, where you can search for and import assets from Poly into your Unity scene directly from the editor.

Poly Toolkit for Unreal Developers

If you are using Unreal, we also offer Poly Toolkit for Unreal, which wraps the API and performs automatic download and conversion of OBJs and Blocks models from Poly. It allows you to query for assets and filter results, download assets and import assets as ready-to-use Unreal actors that you can use in your game.

Credit: Piano by Bruno Oliveira

How to use Poly API with Android, Web or iOS app

Not using a game engine? No problem! If you are developing for Android, check out our Android sample code, which includes a basic sample with no external dependencies, and also a sample that shows how to use the Poly API in conjunction with ARCore. The samples include:

  • Asynchronous HTTP connections to the Poly API.
  • Asynchronous downloading of asset files.
  • Conversion of OBJ and MTL files to OpenGL-compatible VBOs and IBOs.
  • Examples of basic shaders.
  • Integration with ARCore (dynamically downloads an object from Poly and lets the user place it in the scene).

Credit: Cactus wrenby Poly by Google

If you are an iOS developer, we have two samples for you as well: one using SceneKit and one using ARKit, showing how to build an iOS app that downloads and imports models from Poly. This includes all the logic necessary to open an HTTP connection, make the API requests, parse the results, build the 3D objects from the data and place them on the scene.

For web developers, we also offer a complete WebGL sample using Three.js, showing how to get and display a particular asset, or perform searches. There is also a sample showing how to import and display Tilt Brush sketches.

Credit: Forest by Alex "SAFFY" Safayan

No matter what engine or platform you are using, we hope that the Poly API will help bring high quality assets to your app and help you increase engagement with your users! You can find more information about the Poly API and our toolkits and samples on our developers site.

 

4th December 2017 |

Introducing the AIY Vision Kit: Add computer vision to your maker projects
Posted by Billy Rutledge, Director, AIY Projects

Since we released AIY Voice Kit, we've been inspired by the thousands of amazing builds coming in from the maker community. Today, the AIY Team is excited to announce our next project: the AIY Vision Kit — an affordable, hackable, intelligent camera.

Much like the Voice Kit, our Vision Kit is easy to assemble and connects to a Raspberry Pi computer. Based on user feedback, this new kit is designed to work with the smaller Raspberry Pi Zero W computer and runs its vision algorithms on-device so there's no cloud connection required.

Build intelligent devices that can perceive, not just see

The kit materials list includes a VisionBonnet, a cardboard outer shell, an RGB arcade-style button, a piezo speaker, a macro/wide lens kit, flex cables, standoffs, a tripod mounting nut and connecting components.

The VisionBonnet is an accessory board for Raspberry Pi Zero W that features the Intel® Movidius™ MA2450, a low-power vision processing unit capable of running neural networks. This will give makers visual perception instead of image sensing. It can run at speeds of up to 30 frames per second, providing near real-time performance.

Bundled with the software image are three neural network models:

  • A model based on MobileNetsthat can recognize a thousand common objects.
  • A model for face detection capable of not only detecting faces in the image, but also scoring facial expressions on a "joy scale" that ranges from "sad" to "laughing."
  • A model for the important task of discerning between cats, dogs and people.

For those of you who have your own models in mind, we've included the original TensorFlow code and a compiler. Take a new model you have (or train) and run it on the the Intel® Movidius™ MA2450.

Extend the kit to solve your real-world problems

The AIY Vision Kit is completely hackable:

  • Want to prototype your own product? The Vision Kit and the Raspberry Pi Zero W can fit into any number of tiny enclosures.
  • Want to change the way the camera reacts? Use the Python API to write new software to customize the RGB button colors, piezo element sounds and GPIO pins.
  • Want to add more lights, buttons, or servos? Use the 4 GPIO expansion pins to connect your own hardware.

We hope you'll use it to solve interesting challenges, such as:

  • Build "hotdog/not hotdog" (or any other food recognizer)
  • Turn music on when someone walks through the door
  • Send a text when your car leaves the driveway
  • Open the dog door when she wants to get back in the house

Ready to get your hands on one?

AIY Vision Kits will be available in December, with online pre-sales at Micro Center starting today.

*** Please note that AIY Vision Kit requires Raspberry Pi Zero W, Raspberry Pi Camera V2 and a micro SD card, which must be purchased separately.

Tell us what you think!

We're listening — let us know how we can improve our kits and share what you're making using the #AIYProjects hashtag on social media. We hope AIY Vision Kit inspires you to build all kinds of creative devices.

 

1st December 2017 |

Introducing the 5th class of Launchpad Accelerator!
Posted by Roy Glasberg, Global Lead, Google Developers Launchpad

To cap off another amazing year for Launchpad Accelerator, we're excited to announce the 5th class of our hands-on mentorship program. This includes a diverse group of startups from all over the world looking to tackle everything from streamlining medical records in Africato improving breast cancer screenings.

Launchpad Accelerator is Google's six month program that includes an intensive two week bootcamp in San Francisco and mentoring from 30+ teams across Google and expert mentors from top technology companies and VCs in Silicon Valley and globally. Participants receive equity-free support, credits for Google products and media training, and continue to work closely with Google back in their home country.

Class 5 kicks off January 29th, 2018 at the Google Developers Launchpad Space in San Francisco and will include 2 weeks of all-expense-paid training, as part of the the full 6-month program.

Here's the full list of participating startups (by region):

Africa

Nigeria

    Helium Health provides smart, rugged, all-in-one electronic medical records for Africa
    Paylater, by OneFi, is an online provider of digital financial services for the underbanked

South Africa

    Aerobotics helps farmers optimize their yields and reduce their costs through its aerial data analytics platform

Asia

Bangladesh

    Maya Apa is your digital well-being assistant that intelligently understands your question and makes it easy to connect to real doctors & therapists

India

    BabyChakrais the trusted care companion to Indian parents from pregnancy to parenting
    m.Paanipowers real-time, direct to consumer engagement, marketing, loyalty and insights for mass market consumers and retailers
    NIRAMAI is a HealthTech startup that has developed a novel Breast Cancer Screening solution that can save lives by detecting cancer at an early stage
    SocialCopsempowers organizations to make better decisions through data

Indonesia

    Kulina is a meal subscription service that uses technology to optimise inefficiencies in supply chain and logistics

Pakistan

    VividTech transforms customer services experience by allowing users to "dial" into a digital/visual experience of navigating through the IVR trees

Philippines

    Ayannah enables affordable and accessible digital financial services to the world's emerging middle class

Thailand

    Pricezaoffers a search engine and price comparison app for shopping

Vietnam

    Monkey Junior creates educational solutions for kids up to 10 to learn languages, maths and science

Europe

Hungary

    Synetiq helps brands & media producers create high performing video ads using machine learning and biometric emotional insights from their target audience

Poland

    Szopi offers a same-day delivery platform for groceries and pharmaceuticals

Russia

    App in the Air offers a personal travel assistant for flight tracking and exploring airports
    Voximplant offers a versatile cloud communications platform

Latin America

Argentina

    Etermax is a creator of massive social gaming apps such as Trivia Crack
    Restorandois the leading dining-out marketplace in Latin America

Brazil

    Loggicreates new-wave logistics, powered by intelligent software and humane technology
    Nubankis a financial technology company offering a fully digital and branchless experience
    Grupo ZAP Viva Real is an online real estate marketplace that connects buyers, sellers, and renters with properties in Brazil

Mexico

    Clipis the service that allows you to accept payments with any credit or debit card, at any time and anywhere, turning your smartphone or tablet into a banking terminal
    Kubo.financierois a P2P lending platform in Mexico regulated by local authorities including The Bank of México, CNBV and CONDUSEF

 

28th November 2017 |

Actions On Google Best Practices Video Series
Posted by Ido Green (@greenido), Developer Advocate

We recently launched a new YouTube video series focused on teaching developers best practices for the Actions on Google platform.

Apps for the Google Assistant are the gateway for users to engage with your services through Google Home, Android phones, iPhones, and in the future, through every experience where the Google Assistant is available.

The goal of the video series is to show you how to use the Google Assistant platform in the best way. You will learn more from Ido Green, Developer Advocate at Google, who will touch on topics like:

Tune in to learn how to build, or improve your apps for the Google Assistant so your users can benefit from more meaningful, interactive experiences.

And if you'd like to keep the conversation going, please join our developer community at: https://g.co/actionsdev or @actionsongoogle

See you!

 

21st November 2017 |

Introducing container-diff, a tool for quickly comparing container images
Originally posted by Nick Kubala, Colette Torres, and Abby Tisdale from the Container Tools team, on the Google Open Source Blog

The Google Container Tools team originally built container-diff, a new project to help uncover differences between container images, to aid our own development with containers. We think it can be useful for anyone building containerized software, so we're excited to release it as open source to the development community.

Containers and the Dockerfile format help make customization of an application's runtime environment more approachable and easier to understand. While this is a great advantage of using containers in software development, a major drawback is that it can be hard to visualize what changes in a container image will result from a change in the respective Dockerfile. This can lead to bloated images and make tracking down issues difficult.

Imagine a scenario where a developer is working on an application, built on a runtime image maintained by a third-party. During development someone releases a new version of that base image with updated system packages. The developer rebuilds their application and picks up the latest version of the base image, and suddenly their application stops working; it depended on a previous version of one of the installed system packages, but which one? What version was it on before? With no currently existing tool to easily determine what changed between the two base image versions, this totally stalls development until the developer can track down the package version incompatibility.

Introducing container-diff

container-diff helps users investigate image changes by computing semantic diffs between images. What this means is that container-diff figures out on a low-level what data changed, and then combines this with an understanding of package manager information to output this information in a format that's actually readable to users. The tool can find differences in system packages, language-level packages, and files in a container image.

Users can specify images in several formats - from local Docker daemon (using the prefix `daemon://` on the image path), a remote registry (using the prefix `remote://`), or a file in the .tar in the format exported by "docker save" command. You can also combine these formats to compute the diff between a local version of an image and a remote version. This can be useful when experimenting with new builds of an image that you might not be quite ready to push yet. container-diff supports image tarballs and the registry protocol natively, enabling it to run in environments without a Docker daemon.

Examples and Use Cases

Here is a basic Dockerfile that installs Python inside our Debian base image. Running container-diff on the base image and the new one with Python, users can see all the apt packages that were installed as dependencies of Python.

➜  debian_with_python cat Dockerfile
FROM gcr.io/google-appengine/debian8
RUN apt-get update && apt-get install -qq --force-yes python
➜ debian_with_python docker build -q -t debian_with_python .
sha256:be2cd1ae6695635c7041be252589b73d1539a858c33b2814a66fe8fa4b048655
➜ debian_with_python container-diff diff gcr.io/google-appengine/debian8:latest daemon://debian_with_python:latest

-----Apt-----
Packages found only in gcr.io/google-appengine/debian8:latest: None

Packages found only in debian_with_python:latest:
NAME VERSION SIZE
-file 1:5.22 15-2+deb8u3 76K
-libexpat1 2.1.0-6 deb8u4 386K
-libffi6 3.1-2 deb8u1 43K
-libmagic1 1:5.22 15-2+deb8u3 3.1M
-libpython-stdlib 2.7.9-1 54K
-libpython2.7-minimal 2.7.9-2 deb8u1 2.6M
-libpython2.7-stdlib 2.7.9-2 deb8u1 8.2M
-libsqlite3-0 3.8.7.1-1 deb8u2 877K
-mime-support 3.58 146K
-python 2.7.9-1 680K
-python-minimal 2.7.9-1 163K
-python2.7 2.7.9-2 deb8u1 360K
-python2.7-minimal 2.7.9-2 deb8u1 3.7M

Version differences: None

And below is a Dockerfile that inherits from our Python base runtime image, and then installs the mock and six packages inside of it. Running container-diff with the pip differ, users can see all the Python packages that have either been installed or changed as a result of this:

➜  python_upgrade cat Dockerfile
FROM gcr.io/google-appengine/python
RUN pip install -U six
➜ python_upgrade docker build -q -t python_upgrade .
sha256:7631573c1bf43727d7505709493151d3df8f98c843542ed7b299f159aec6f91f
➜ python_upgrade container-diff diff gcr.io/google-appengine/python:latest daemon://python_upgrade:latest --types=pip

-----Pip-----

Packages found only in gcr.io/google-appengine/python:latest: None

Packages found only in python_upgrade:latest:
NAME VERSION SIZE
-funcsigs 1.0.2 51.4K
-mock 2.0.0 531.2K
-pbr 3.1.1 471.1K

Version differences:
PACKAGE IMAGE1 (gcr.io/google-appengine/python:latest) IMAGE2 (python_upgrade:latest)
-six 1.8.0, 26.7K

This can be especially useful when it's unclear which packages might have been installed or changed incidentally as a result of dependency management of Python modules.

These are just a few examples. The tool currently has support for Python and Node.js packages installed via pip and npm, respectively, as well as comparison of image filesystems and Docker history. In the future, we'd like to see support added for additional runtime and language differs, including Java, Go, and Ruby. External contributions are welcome! For more information on contributing to container-diff, see this how-to guide.

Now that we've seen container-diff compare two images in action, it's easy to imagine how the tool may be integrated into larger workflows to aid in development:

  • Changelog generation: Given container-diff's capacity to facilitate investigation of filesystem and package modifications, it can do most of the heavy lifting in discerning changes for automatic changelog generation for new releases of an image.
  • Continuous integration: As part of a CI system, users can leverage container-diff to catch potentially breaking filesystem changes resulting from a Dockerfile change in their builds.

container-diff's default output mode is "human-readable," but also supports output to JSON, allowing for easy automated parsing and processing by users.

Single Image Analysis

In addition to comparing two images, container-diff has the ability to analyze a single image on its own. This can enable users to get a quick glance at information about an image, such as its system and language-level package installations and filesystem contents.

Let's take a look at our Debian base image again. We can use the tool to easily view a list of all packages installed in the image, along with each one's installed version and size:

➜  Development container-diff analyze gcr.io/google-appengine/debian8:latest

-----Apt-----
Packages found in gcr.io/google-appengine/debian8:latest:
NAME VERSION SIZE
-acl 2.2.52-2 258K
-adduser 3.113 nmu3 1M
-apt 1.0.9.8.4 3.1M
-base-files 8 deb8u9 413K
-base-passwd 3.5.37 185K
-bash 4.3-11 deb8u1 4.9M
-bsdutils 1:2.25.2-6 181K
-ca-certificates 20141019 deb8u3 367K
-coreutils 8.23-4 13.9M
-dash 0.5.7-4 b1 191K
-debconf 1.5.56 deb8u1 614K
-debconf-i18n 1.5.56 deb8u1 1.1M
-debian-archive-keyring 2017.5~deb8u1 137K

We could use this to verify compatibility with an application we're building, or maybe sort the packages by size in another one of our images and see which ones are taking up the most space.

For more information about this tool as well as a breakdown with examples, uses, and inner workings of the tool, please take a look at documentation on our GitHub page. Happy diffing!

Special thanks to Colette Torres and Abby Tisdale, our software engineering interns who helped build the tool from the ground up.

 

21st November 2017 |

Introducing TensorFlow Feature Columns
Posted by the TensorFlow Team

Welcome to Part 2 of a blog series that introduces TensorFlow Datasets and Estimators. We're devoting this article to feature columns—a data structure describing the features that an Estimator requires for training and inference. As you'll see, feature columns are very rich, enabling you to represent a diverse range of data.

In Part 1, we used the pre-made Estimator DNNClassifier to train a model to predict different types of Iris flowers from four input features. That example created only numerical feature columns (of type tf.feature_column.numeric_column). Although those feature columns were sufficient to model the lengths of petals and sepals, real world data sets contain all kinds of non-numerical features. For example:

Figure 1. Non-numerical features.

How can we represent non-numerical feature types? That's exactly what this blogpost is all about.

Input to a Deep Neural Network

Let's start by asking what kind of data can we actually feed into a deep neural network? The answer is, of course, numbers (for example, tf.float32). After all, every neuron in a neural network performs multiplication and addition operations on weights and input data. Real-life input data, however, often contains non-numerical (categorical) data. For example, consider a product_class feature that can contain the following three non-numerical values:

  • kitchenware
  • electronics
  • sports

ML models generally represent categorical values as simple vectors in which a 1 represents the presence of a value and a 0 represents the absence of a value. For example, when product_class is set to sports, an ML model would usually represent product_class as [0, 0, 1], meaning:

  • 0: kitchenware is absent
  • 0: electronics is absent
  • 1: sports: is present

So, although raw data can be numerical or categorical, an ML model represents all features as either a number or a vector of numbers.

Introducing Feature Columns

As Figure 2 suggests, you specify the input to a model through the feature_columns argument of an Estimator (DNNClassifier for Iris). Feature Columns bridge input data (as returned by input_fn) with your model.

Figure 2. Feature columns bridge raw data with the data your model needs.

To represent features as a feature column, call functions of the tf.feature_columnpackage. This blogpost explains nine of the functions in this package. As Figure 3 shows, all nine functions return either a Categorical-Column or a Dense-Column object, except bucketized_column which inherits from both classes:

Figure 3. Feature column methods fall into two main categories and one hybrid category.

Let's look at these functions in more detail.

Numeric Column

The Iris classifier called tf.numeric_column()for all input features: SepalLength, SepalWidth, PetalLength, PetalWidth. Although tf.numeric_column() provides optional arguments, calling the function without any arguments is a perfectly easy way to specify a numerical value with the default data type (tf.float32) as input to your model. For example:

# Defaults to a tf.float32 scalar.
numeric_feature_column = tf.feature_column.numeric_column(key="SepalLength")

Use the dtype argument to specify a non-default numerical data type. For example:

# Represent a tf.float64 scalar.
numeric_feature_column = tf.feature_column.numeric_column(key="SepalLength",
dtype=tf.float64)

By default, a numeric column creates a single value (scalar). Use the shape argument to specify another shape. For example:

# Represent a 10-element vector in which each cell contains a tf.float32.
vector_feature_column = tf.feature_column.numeric_column(key="Bowling",
shape=10)

# Represent a 10x5 matrix in which each cell contains a tf.float32.
matrix_feature_column = tf.feature_column.numeric_column(key="MyMatrix",
shape=[10,5])

Bucketized Column

Often, you don't want to feed a number directly into the model, but instead split its value into different categories based on numerical ranges. To do so, create a bucketized column. For example, consider raw data that represents the year a house was built. Instead of representing that year as a scalar numeric column, we could split year into the following four buckets:

Figure 4. Dividing year data into four buckets.

The model will represent the buckets as follows:

 Date Range  Represented as...
 < 1960  [1, 0, 0, 0]
 >= 1960 but < 1980   [0, 1, 0, 0]
 >= 1980 but < 2000   [0, 0, 1, 0]
 > 2000  [0, 0, 0, 1]

Why would you want to split a number—a perfectly valid input to our model—into a categorical value like this? Well, notice that the categorization splits a single input number into a four-element vector. Therefore, the model now can learn four individual weights rather than just one. Four weights creates a richer model than one. More importantly, bucketizing enables the model to clearly distinguish between different year categories since only one of the elements is set (1) and the other three elements are cleared (0). When we just use a single number (a year) as input, the model can't distinguish categories. So, bucketing provides the model with additional important information that it can use to learn.

The following code demonstrates how to create a bucketized feature:

# A numeric column for the raw input.
numeric_feature_column = tf.feature_column.numeric_column("Year")

# Bucketize the numeric column on the years 1960, 1980, and 2000
bucketized_feature_column = tf.feature_column.bucketized_column(
source_column = numeric_feature_column,
boundaries = [1960, 1980, 2000])

Note the following:

  • Before creating the bucketized column, we first created a numeric column to represent the raw year.
  • We passed the numeric column as the first argument to tf.feature_column.bucketized_column().
  • Specifying a three-element boundaries vector creates a four-element bucketized vector.

Categorical identity column

Categorical identity columns are a special case of bucketized columns. In traditional bucketized columns, each bucket represents a range of values (for example, from 1960 to 1979). In a categorical identity column, each bucket represents a single, unique integer. For example, let's say you want to represent the integer range [0, 4). (That is, you want to represent the integers 0, 1, 2, or 3.) In this case, the categorical identity mapping looks like this:

Figure 5. A categorical identity column mapping. Note that this is a one-hot encoding, not a binary numerical encoding.

So, why would you want to represent values as categorical identity columns? As with bucketized columns, a model can learn a separate weight for each class in a categorical identity column. For example, instead of using a string to represent the product_class, let's represent each class with a unique integer value. That is:

  • 0="kitchenware"
  • 1="electronics"
  • 2="sport"

Call tf.feature_column.categorical_column_with_identity()to implement a categorical identity column. For example:

# Create a categorical output for input "feature_name_from_input_fn",
# which must be of integer type. Value is expected to be >= 0 and < num_buckets
identity_feature_column = tf.feature_column.categorical_column_with_identity(
key='feature_name_from_input_fn',
num_buckets=4) # Values [0, 4)

# The 'feature_name_from_input_fn' above needs to match an integer key that is
# returned from input_fn (see below). So for this case, 'Integer_1' or
# 'Integer_2' would be valid strings instead of 'feature_name_from_input_fn'.
# For more information, please check out Part 1 of this blog series.
def input_fn():
...<code>...
return ({ 'Integer_1':[values], ..<etc>.., 'Integer_2':[values] },
[Label_values])

Categorical vocabulary column

We cannot input strings directly to a model. Instead, we must first map strings to numeric or categorical values. Categorical vocabulary columns provide a good way to represent strings as a one-hot vector. For example:

Figure 6. Mapping string values to vocabulary columns.

As you can see, categorical vocabulary columns are kind of an enum version of categorical identity columns. TensorFlow provides two different functions to create categorical vocabulary columns:

The tf.feature_column.categorical_column_with_vocabulary_list() function maps each string to an integer based on an explicit vocabulary list. For example:

# Given input "feature_name_from_input_fn" which is a string,
# create a categorical feature to our model by mapping the input to one of
# the elements in the vocabulary list.
vocabulary_feature_column =
tf.feature_column.categorical_column_with_vocabulary_list(
key="feature_name_from_input_fn",
vocabulary_list=["kitchenware", "electronics", "sports"])

The preceding function has a significant drawback; namely, there's way too much typing when the vocabulary list is long. For these cases, call tf.feature_column.categorical_column_with_vocabulary_file()instead, which lets you place the vocabulary words in a separate file. For example:

# Given input "feature_name_from_input_fn" which is a string,
# create a categorical feature to our model by mapping the input to one of
# the elements in the vocabulary file
vocabulary_feature_column =
tf.feature_column.categorical_column_with_vocabulary_file(
key="feature_name_from_input_fn",
vocabulary_file="product_class.txt",
vocabulary_size=3)

# product_class.txt should have one line for vocabulary element, in our case:
kitchenware
electronics
sports

Using hash buckets to limit categories

So far, we've worked with a naively small number of categories. For example, our product_class example has only 3 categories. Often though, the number of categories can be so big that it's not possible to have individual categories for each vocabulary word or integer because that would consume too much memory. For these cases, we can instead turn the question around and ask, "How many categories am I willing to have for my input?" In fact, the tf.feature_column.categorical_column_with_hash_buckets()function enables you to specify the number of categories. For example, the following code shows how this function calculates a hash value of the input, then puts it into one of the hash_bucket_size categories using the modulo operator:

# Create categorical output for input "feature_name_from_input_fn".
# Category becomes: hash_value("feature_name_from_input_fn") % hash_bucket_size
hashed_feature_column =
tf.feature_column.categorical_column_with_hash_bucket(
key = "feature_name_from_input_fn",
hash_buckets_size = 100) # The number of categories

At this point, you might rightfully think: "This is crazy!" After all, we are forcing the different input values to a smaller set of categories. This means that two, probably completely unrelated inputs, will be mapped to the same category, and consequently mean the same thing to the neural network. Figure 7 illustrates this dilemma, showing that kitchenware and sports both get assigned to category (hash bucket) 12:

Figure 7. Representing data in hash buckets.

As with many counterintuitive phenomena in machine learning, it turns out that hashing often works well in practice. That's because hash categories provide the model with some separation. The model can use additional features to further separate kitchenware from sports.

Feature crosses

The last categorical column we'll cover allows us to combine multiple input features into a single one. Combining features, better known as feature crosses, enables the model to learn separate weights specifically for whatever that feature combination means.

More concretely, suppose we want our model to calculate real estate prices in Atlanta, GA. Real-estate prices within this city vary greatly depending on location. Representing latitude and longitude as separate features isn't very useful in identifying real-estate location dependencies; however, crossing latitude and longitude into a single feature can pinpoint locations. Suppose we represent Atlanta as a grid of 100x100 rectangular sections, identifying each of the 10,000 sections by a cross of its latitude and longitude. This cross enables the model to pick up on pricing conditions related to each individual section, which is a much stronger signal than latitude and longitude alone.

Figure 8 shows our plan, with the latitude & longitude values for the corners of the city:

Figure 8. Map of Atlanta. Imagine this map divided into 10,000 sections of equal size.

For the solution, we used a combination of some feature columns we've looked at before, as well as the tf.feature_columns.crossed_column()function.

# In our input_fn, we convert input longitude and latitude to integer values
# in the range [0, 100)
def input_fn():
# Using Datasets, read the input values for longitude and latitude
latitude = ... # A tf.float32 value
longitude = ... # A tf.float32 value

# In our example we just return our lat_int, long_int features.
# The dictionary of a complete program would probably have more keys.
return { "latitude": latitude, "longitude": longitude, ...}, labels

# As can be see from the map, we want to split the latitude range
# [33.641336, 33.887157] into 100 buckets. To do this we use np.linspace
# to get a list of 99 numbers between min and max of this range.
# Using this list we can bucketize latitude into 100 buckets.
latitude_buckets = list(np.linspace(33.641336, 33.887157, 99))
latitude_fc = tf.feature_column.bucketized_column(
tf.feature_column.numeric_column('latitude'),
latitude_buckets)

# Do the same bucketization for longitude as done for latitude.
longitude_buckets = list(np.linspace(-84.558798, -84.287259, 99))
longitude_fc = tf.feature_column.bucketized_column(
tf.feature_column.numeric_column('longitude'), longitude_buckets)

# Create a feature cross of fc_longitude x fc_latitude.
fc_san_francisco_boxed = tf.feature_column.crossed_column(
keys=[latitude_fc, longitude_fc],
hash_bucket_size=1000) # No precise rule, maybe 1000 buckets will be good?

You may create a feature cross from either of the following:

  • Feature names; that is, names from the dict returned from input_fn.
  • Any Categorical Column (see Figure 3), except categorical_column_with_hash_bucket.

When feature columns latitude_fc and longitude_fc are crossed, TensorFlow will create 10,000 combinations of (latitude_fc, longitude_fc) organized as follows:

(0,0),(0,1)...  (0,99)
(1,0),(1,1)... (1,99)
…, …, ...
(99,0),(99,1)...(99, 99)

The function tf.feature_column.crossed_column performs a hash calculation on these combinations and then slots the result into a category by performing a modulo operation with hash_bucket_size. As discussed before, performing the hash and modulo function will probably result in category collisions; that is, multiple (latitude, longitude) feature crosses will end up in the same hash bucket. In practice though, performing feature crosses still provides significant value to the learning capability of your models.

Somewhat counterintuitively, when creating feature crosses, you typically still should include the original (uncrossed) features in your model. For example, provide not only the (latitude, longitude) feature cross but also latitude and longitude as separate features. The separate latitude and longitude features help the model separate the contents of hash buckets containing different feature crosses.

See this link for a full code example for this. Also, the reference section at the end of this post for lots more examples of feature crossing.

Indicator and embedding columns

Indicator columns and embedding columns never work on features directly, but instead take categorical columns as input.

When using an indicator column, we're telling TensorFlow to do exactly what we've seen in our categorical product_class example. That is, an indicator column treats each category as an element in a one-hot vector, where the matching category has value 1 and the rest have 0s:

Figure 9. Representing data in indicator columns.

Here's how you create an indicator column:

categorical_column = ... # Create any type of categorical column, see Figure 3

# Represent the categorical column as an indicator column.
# This means creating a one-hot vector with one element for each category.
indicator_column = tf.feature_column.indicator_column(categorical_column)

Now, suppose instead of having just three possible classes, we have a million. Or maybe a billion. For a number of reasons (too technical to cover here), as the number of categories grow large, it becomes infeasible to train a neural network using indicator columns.

We can use an embedding column to overcome this limitation. Instead of representing the data as a one-hot vector of many dimensions, anembedding column represents that data as a lower-dimensional, ordinary vector in which each cell can contain any number, not just 0 or 1. By permitting a richer palette of numbers for every cell, an embedding column contains far fewer cells than an indicator column.

Let's look at an example comparing indicator and embedding columns. Suppose our input examples consists of different words from a limited palette of only 81 words. Further suppose that the data set provides the following input words in 4 separate examples:

  • "dog"
  • "spoon"
  • "scissors"
  • "guitar"

In that case, Figure 10 illustrates the processing path for embedding columns or Indicator columns.

Figure 10. An embedding column stores categorical data in a lower-dimensional vector than an indicator column. (We just placed random numbers into the embedding vectors; training determines the actual numbers.)

When an example is processed, one of the categorical_column_with...functions maps the example string to a numerical categorical value. For example, a function maps "spoon" to [32]. (The 32comes from our imagination—the actual values depend on the mapping function.) You may then represent these numerical categorical values in either of the following two ways:

  • As an indicator column. A function converts each numeric categorical value into an 81-element vector (because our palette consists of 81 words), placing a 1 in the index of the categorical value (0, 32, 79, 80) and a 0 in all the other positions.
  • As an embedding column. A function uses the numerical categorical values (0, 32, 79, 80) as indices to a lookup table. Each slot in that lookup table contains a 3-element vector.

How do the values in the embeddings vectors magically get assigned? Actually, the assignments happen during training. That is, the model learns the best way to map your input numeric categorical values to the embeddings vector value in order to solve your problem. Embedding columns increase your model's capabilities, since an embeddings vector learns new relationships between categories from the training data.

Why is the embedding vector size 3 in our example? Well, the following "formula" provides a general rule of thumb about the number of embedding dimensions:

embedding_dimensions =  number_of_categories**0.25

That is, the embedding vector dimension should be the 4th root of the number of categories. Since our vocabulary size in this example is 81, the recommended number of dimensions is 3:

3 =  81**0.25

Note that this is just a general guideline; you can set the number of embedding dimensions as you please.

Call tf.feature_column.embedding_columnto create an embedding_column. The dimension of the embedding vector depends on the problem at hand as described above, but common values go as low as 3 all the way to 300 or even beyond:

categorical_column = ... # Create any categorical column shown in Figure 3.

# Represent the categorical column as an embedding column.
# This means creating a one-hot vector with one element for each category.
embedding_column = tf.feature_column.embedding_column(
categorical_column=categorical_column,
dimension=dimension_of_embedding_vector)

Embeddings is a big topic within machine learning. This information was just to get you started using them as feature columns. Please see the end of this post for more information.

Passing feature columns to Estimators

Still there? I hope so, because we only have a tiny bit left before you've graduated from the basics of feature columns.

As we saw in Figure 1, feature columns map your input data (described by the feature dictionary returned from input_fn) to values fed to your model. You specify feature columns as a list to a feature_columns argument of an estimator. Note that the feature_columns argument(s) vary depending on the Estimator:

The reason for the above rules are beyond the scope of this introductory post, but we will make sure to cover it in a future blogpost.

Summary

Use feature columns to map your input data to the representations you feed your model. We only used numeric_column in Part 1 of this series , but working with the other functions described in this post, you can easily create other feature columns.

For more details on feature columns, be sure to check out:

If you want to learn more about embeddings:

 

17th November 2017 |

Introducing our new developer YouTube Series: “Build Out”
Posted by Reto Meier & Colt McAnlis: Developer Advocates

Ever found yourself trying to figure out the right way to combine mobile, cloud, and web technologies, only to be lost in the myriad of available offerings? It can be challenging to know the best way to combine all the options to build products that solve problems for your users.

That's why we created Build Out, a new YouTube series where real engineers face-off building fake products.

Each month we, (Reto Meier and Colt McAnlis), will present competing architectures to help show how Google's developer products can be combined to solve challenging problems for your users. Each solution incorporates a wide range of technologies, including Google Cloud, Android, Firebase, and Tensorflow (just to name a few).

Since we're engineers at heart, we enjoy a challenge—so each solution goes well past minimum viable product, and explores some of the more advanced possibilities available to solve the problem creatively.

Now, here's the interesting part. When we're done presenting, you get to decide which of us solved the problem better, by posting a comment to the video on YouTube. If you've already got a better solution—or think you know one—tell us about it in the comments, or respond with your own Build Out video to show us how it's done!

Episode #1: The Smart Garden.

In which we explore designs for gardens that care for themselves. Each design must be fully autonomous, learn from experience, and scale from backyard up to large-scale commercial gardens.

You can get the full technical details on each Smart Garden solution in this Medium article, including alternative approaches and best practices.

You can also listen to the Build Out Rewound Podcast, to hear us discuss our choices.

 

16th November 2017 |

Launchpad comes to Africa to support tech startups! Apply to join the first accelerator class
Posted by Andy Volk, Sub-Saharan Africa Ecosystem Regional Manager & Josh Yellin, Program Manager of Launchpad Accelerator

Earlier this year at Google for Nigeria, our CEO Sundar Pichai made a commitmentto support African entrepreneurs building successful technology companies and products. Following up on that commitment, we're excited to announce Google Developers Launchpad Africa , our new hands-on comprehensive mentorship program tailored exclusively to startups based in Africa.

Building on the success of our global Launchpad Accelerator program, Launchpad Africa will kick-off as a three-month accelerator that provides African startups with over $3 million in equity-free support, working space, travel and PR backing, and access to expert advisers from Google, Africa, and around the world.

The first applicationperiod is now open through December 11, 9am PST and the first class will start in early 2018. More classes will be hosted in 2018 and beyond.

What do we look for when selecting startups?

Each startup that applies to Launchpad Africa is evaluated carefully. Below are general guidelines behind our process to help you understand what we look for in our candidates.

All startups in the program must:

  • Be a technology startup.
  • Be based in Ghana, Kenya, Nigeria, South Africa, Tanzania, or Uganda (stay tuned for future classes, as we hope to add more countries).
  • Have already raised seed funding.

Additionally, we also consider:

  • The problem you're trying to solve. How does it create value for users? How are you addressing a real challenge for your home city, country, or Africa broadly?
  • Will you share what you learn for the benefit of other startups in your local ecosystem?

Anyone who spends time in the African technology space knows that the continent is home to some exciting innovations. Over the years, Google has worked with some incredible startups across Africa, tackling everything from healthcare, education, streamlining e-commerce, to improving the food supply chain. We very much look forward to welcoming the first cohort of innovators for Launchpad Africa and continue to work together to drive innovation in the African market.

 

21st November 2017 |

Help users find, interact & re-engage with your app on the Google Assistant
Posted by Brad Abrams, Product Manager
Every day, users are discovering new ways the Google Assistant and your apps can help them get things done. Today we're announcing a set of new features to make it easier for users to find, interact, and re-engage with your app.

Helping users find your apps

With more international support and updates to the Google Assistant, it's easier than ever for users to find your app.
  • Updates to the app directory: We're adding what's new and what's trending sections in the app directory within the Assistant experience on your phone. These dynamic sections will constantly change and evolve, creating more opportunities for your app to be discovered by users in all supported locales where the Google Assistant and Actions on Google are available. We're also introducing autocomplete in the directory's search box, so, if a user doesn't quite remember the name of your app, it will populate as they type.
  • New subcategories: We've created subcategories in the app directory, so if you click on a category like "Food & Drink", apps are broken down into additional subcategories, like "Order Food" or "View a Menu." We're using your app's description and sample invocations to map users' natural search queries to the new task-based subcategories. The updated labelling taxonomy improves discovery for your app; it will now surface for users in all relevant subcategories depending on its various capabilities. This change will help you communicate to users everything your app can do, and creates new avenues for your app to be discovered – learn more here.
  • Implicit discovery: Implicit discovery is when a user is connected to your app using contextual queries (e.g., "book an appointment to fix my bike"), as opposed to calling for your app by name. We've created a new discovery section of the console to help improve your app's implicit discovery, providing instructions for creating precise action invocation phrases so your app will surface even when a user can't remember its name. Go hereto learn more.
  • Badges for family-friendly apps: We're launching a new "For Families" badge on the Google Assistant, designed to help users find apps that are appropriate for all ages. All existing apps in the Apps for Families program will get the badge automatically. Learn about how your app can qualify for the "For Families" badge here.
  • International support: Users will soon be able to find your apps in even more languages because starting today, you can build apps in Spanish (US, MX and ES), Italian, Portuguese (BR) and English (IN). And in the UK, developers can now start building apps that have transactional capabilities. Watch the internationalization videoto learn how to support multiple languages with Actions on Google.

Creating a more interactive user experience

Helping users find your app is one thing, but making sure they have a compelling, meaningful experience once they begin talking to your app is equally important – we're releasing some new features to help:
  • Speaker to phone transfer: We're launching a new API so you can develop experiences that start with the Assistant on voice-activated speakers like Google Home and can be passed off to users' phones. Need to send a map or complete a transaction using a phone? Check out the example below and click hereto learn more.
  • Build personalized apps: To create a more personal experience for users, you can now enable your app to remember select information and preferences. Learn more here.
  • Better SSML: We recently rolled out an update to the web simulator which includes a new SSML audio design experience. We now give you more options for creating natural, quality dialog using newly supported SSML tags, including <prosody>, <emphasis>, <audio> and others. The new tag <par> is coming soon and lets you add mood and richness, so you can play background music and ambient sounds while a user is having a conversation with your app. To help you get started, we've added over 1,000 sounds to the sound library.Listen to a brief SSML audio experiment that shows off some of the new features here 🔊.
  • Cancel event: Today when a user says "cancel" to end the conversation, your app never gets a chance to respond with a polite farewell message. Now you can get one last request to your webhook that you can use to clean up your fulfillment logic and respond to the user before they exit.
  • Account linking in conversation: Until today, users had to link their account to your app at the beginning of the interaction, before they had a chance to decide whether or not account linking was the right choice. With the updated AskForSignInAPI, we're giving you the option of prompting users to link their account to your app at the most appropriate time of the experience.

Re-engaging with your users

To keep users coming back to your app, day after day, we're adding some additional features that you can experiment with – these are available this week for you to start testing and will roll out to users soon.
  • Daily updates: At the end of a great interaction with your app, a user might want to be notified of similar content from your app every day. To enable that we will add a suggestion chip prompting the user to sign up for a daily update. Check out the example below and go to the discovery section of the console to configure daily updates.
  • Push notifications: We're launching a new push notification API, enabling your app to push asynchronous updates to users. For the day trader who's looking for the best time to sell stock options, or the frugal shopper waiting for the big sale to buy a new pair of shoes, these alerts will show up as system notifications on the phone (and later to the Assistant on voice-activated speakers like Google Home).
  • Directory analytics: To give you more insight into how users are interacting with your app on the mobile directory so you can continue improving the experience for users, we've updated the analytics tools in the console. You will be able to find information about your app's rating, the number of pageviews, along with the number of conversations that were initiated from your app directory listing.
Phew! I know that was a lot to cover, but that was only a brief overview of the updates we've made and we can't wait to see how you'll use these tools to unlock the Google Assistant's potential in new and creative ways.

 

7th December 2017 |

Announcing TensorFlow Lite
Posted by the TensorFlow team
Today, we're happy to announce the developer preview of TensorFlow Lite, TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny IoT devices, but as the adoption of machine learning models has grown exponentially over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models.

It is designed from scratch to be:
  • Lightweight Enables inference of on-device machine learning models with a small binary size and fast initialization/startup
  • Cross-platform A runtime designed to run on many different platforms, starting with Android and iOS
  • Fast Optimized for mobile devices, including dramatically improved model loading times, and supporting hardware acceleration
More and more mobile devices today incorporate purpose-built custom hardware to process ML workloads more efficiently. TensorFlow Lite supports the Android Neural Networks API to take advantage of these new accelerators as they come available.
TensorFlow Lite falls back to optimized CPU execution when accelerator hardware is not available, which ensures your models can still run fast on a large set of devices.

Architecture

The following diagram shows the architectural design of TensorFlow Lite:
The individual components are:
  • TensorFlow Model: A trained TensorFlow model saved on disk.
  • TensorFlow Lite Converter: A program that converts the model to the TensorFlow Lite file format.
  • TensorFlow Lite Model File: A model file format based on FlatBuffers, that has been optimized for maximum speed and minimum size.
The TensorFlow Lite Model File is then deployed within a Mobile App, where:
  • Java API: A convenience wrapper around the C++ API on Android
  • C++ API: Loads the TensorFlow Lite Model File and invokes the Interpreter. The same library is available on both Android and iOS
  • Interpreter: Executes the model using a set of operators. The interpreter supports selective operator loading; without operators it is only 70KB, and 300KB with all the operators loaded. This is a significant reduction from the 1.5M required by TensorFlow Mobile (with a normal set of operators).
  • On select Android devices, the Interpreter will use the Android Neural Networks API for hardware acceleration, or default to CPU execution if none are available.
Developers can also implement custom kernels using the C++ API, that can be used by the Interpreter.

Models

TensorFlow Lite already has support for a number of models that have been trained and optimized for mobile:
  • MobileNet: A class of vision models able to identify across 1000 different object classes, specifically designed for efficient execution on mobile and embedded devices
  • Inception v3: An image recognition model, similar in functionality to MobileNet, that offers higher accuracy but also has a larger size
  • Smart Reply: An on-device conversational model that provides one-touch replies to incoming conversational chat messages. First-party and third-party messaging apps use this feature on Android Wear.
Inception v3 and MobileNets have been trained on the ImageNet dataset. You can easily retrain these on your own image datasets through transfer learning.

What About TensorFlow Mobile?

As you may know, TensorFlow already supports mobile and embedded deployment of models through the TensorFlow Mobile API. Going forward, TensorFlow Lite should be seen as the evolution of TensorFlow Mobile, and as it matures it will become the recommended solution for deploying models on mobile and embedded devices. With this announcement, TensorFlow Lite is made available as a developer preview, and TensorFlow Mobile is still there to support production apps.
The scope of TensorFlow Lite is large and still under active development. With this developer preview, we have intentionally started with a constrained platform to ensure performance on some of the most important common models. We plan to prioritize future functional expansion based on the needs of our users. The goals for our continued development are to simplify the developer experience, and enable model deployment for a range of mobile and embedded devices.
We are excited that developers are getting their hands on TensorFlow Lite. We plan to support and address our external community with the same intensity as the rest of the TensorFlow project. We can't wait to see what you can do with TensorFlow Lite.
For more information, check out the TensorFlow Lite documentation pages.
Stay tuned for more updates.
Happy TensorFlow Lite coding!

 

15th November 2017 |

Reminder: Grow with Google scholarship window closes soon
Posted by Peter Lubbers, Head of Google Developer Training

Last month, we announced the 50,000 Grow with Google scholarship challenge in partnership with Udacity. And today, we want to remind you to apply for the programs before the application window closes on November 30th.

In case you missed the announcement details, the Google-Udacity curriculum was created to help developers get the training they need to enter the workforce as Android or mobile web developers. Whether you're an experienced programmer looking for a career-change or a novice looking for a start, the courses and the Nanodegree programs are built with your career-goals in mind and prepare you for Google's Associate Android Developer and Mobile Web Specialist developer certifications.

The scholarship challenge is an exciting chance to learn valuable skills to launch or advance your career as a mobile or web developer. The program leverages world-class curriculum, developed by experts from Google and Udacity. These courses are completely free, and as a reminder the top 5,000 students at the end of the challenge will earn a full Nanodegree scholarship to one of the four Nanodegree programs in Android or web development.

To learn more visit udacity.com/grow-with-google and submit your application before the scholarship window closes!

 

10th November 2017 |

Best practices to succeed with Universal App campaigns
Posted by Sissie Hsiao, VP of Product, Mobile App Advertising

It's almost time to move all your AdWords app install campaigns to Universal App campaigns (UAC). Existing Search, Display and YouTube app promo campaigns will stop running on November 15th, so it's important to start upgrading to UAC as soon as possible.

With UAC, you can reach the right people across all of Google's largest properties like Google Search, Google Play, YouTube and the Google Display Network — all from one campaign. Marketers who are already using UAC to optimize in-app actions are seeing 140% more conversions per dollar, on average, than other Google app promotion products.1

One of my favorite apps, Maven, a car sharing service from General Motors (GM), is already seeing great success with UAC. According to Kristen Alexander, Marketing Manager: "Maven believes in connecting people with the moments that matter to them. This car sharing audience is largely urban millennials and UAC helps us find this unique, engaged audience across the scale of Google. UAC for Actions helped us increase monthly Android registrations in the Maven app by 51% between April and June."

Join Kristen and others who are already seeing better results with UAC by following some best practices, which I've shared in these blog posts:

Steer Performance with Goals Create a separate UAC for each type of app user that you'd like to acquire — whether that's someone who will install your app or someone who will perform an in-app action after they've installed. Then increase the daily campaign budget for the UAC that's more important right now.

Optimize for the Right In-app Action

Track all important conversion events in your app to learn how users engage with it. Then pick an in-app action that's valuable to your business and is completed by at least 10 different people every day. This will give UAC enough data to find more users who will most likely complete the same in-app action.

Steer Performance with Creative Assets

Supply a healthy mix of creative assets (text, images and videos) that UAC can use to build ads optimized for your goal. Then use the Creative Asset Report to identify which assets are performing "Best" and which ones you should replace.

Follow these and other best practices to help you get positive results from your Universal App campaigns once you upgrade.

Notes


  1. Google Internal Data, July 2017