Includes index

Introduction 1 About This Book 1 Foolish Assumptions 2 Icons Used in This Book 3 Beyond the Book 4 Where to Go from Here 5 Part 1: Discovering Deep Learning 7 Chapter 1: Introducing Deep Learning 9 Defining What Deep Learning Means 10 Starting from Artificial Intelligence 10 Considering the role of AI 12 Focusing on machine learning 15 Moving from machine learning to deep learning 16 Using Deep Learning in the Real World 18 Understanding the concept of learning 18 Performing deep learning tasks 19 Employing deep learning in applications 19 Considering the Deep Learning Programming Environment 19 Overcoming Deep Learning Hype 22 Discovering the start-up ecosystem 22 Knowing when not to use deep learning 22 Chapter 2: Introducing the Machine Learning Principles 25 Defining Machine Learning 26 Understanding how machine learning works 26 Understanding that it's pure math 27 Learning by different strategies 28 Training, validating, and testing data 30 Looking for generalization 31 Getting to know the limits of bias 32 Keeping model complexity in mind 33 Considering the Many Different Roads to Learning 33 Understanding there is no free lunch 34 Discovering the five main approaches 34 Delving into some different approaches 36 Awaiting the next breakthrough 40 Pondering the True Uses of Machine Learning 40 Understanding machine learning benefits 41 Discovering machine learning limits 43 Chapter 3: Getting and Using Python 45 Working with Python in this Book 46 Obtaining Your Copy of Anaconda 46 Getting Continuum Analytics Anaconda 47 Installing Anaconda on Linux 47 Installing Anaconda on MacOS 48 Installing Anaconda on Windows 49 Downloading the Datasets and Example Code 54 Using Jupyter Notebook 54 Defining the code repository 56 Getting and using datasets 61 Creating the Application 62 Understanding cells 62 Adding documentation cells 63 Using other cell types 64 Understanding the Use of Indentation 65 Adding Comments 66 Understanding comments 67 Using comments to leave yourself reminders 68 Using comments to keep code from executing 69 Getting Help with the Python Language 69 Working in the Cloud 70 Using the Kaggle datasets and kernels 70 Using the Google Colaboratory 70 Chapter 4: Leveraging a Deep Learning Framework 73 Presenting Frameworks 74 Defining the differences 74 Explaining the popularity of frameworks 75 Defining the deep learning framework 77 Choosing a particular framework 78 Working with Low-End Frameworks 79 Caffe2 79 Chainer 80 PyTorch 80 MXNet 81 Microsoft Cognitive Toolkit/CNTK 82 Understanding TensorFlow 82 Grasping why TensorFlow is so good 82 Making TensorFlow easier by using TFLearn 84 Using Keras as the best simplifier 85 Getting your copy of TensorFlow and Keras 86 Fixing the C++ build tools error in Windows 88 Accessing your new environment in Notebook 89 Part 2: Considering Deep Learning Basics 91 Chapter 5: Reviewing Matrix Math and Optimization 93 Revealing the Math You Really Need 94 Working with data 94 Creating and operating with a matrix 95 Understanding Scalar, Vector, and Matrix Operations 96 Creating a matrix 97 Performing matrix multiplication 99 Executing advanced matrix operations 100 Extending analysis to tensors 102 Using vectorization effectively 104 Interpreting Learning as Optimization 105 Exploring cost functions 105 Descending the error curve 106 Learning the right direction 107 Updating 109 Chapter 6: Laying Linear Regression Foundations 111 Combining Variables 112 Working through simple linear regression 112 Advancing to multiple linear regression 113 Including gradient descent 115 Seeing linear regression in action 116 Mixing Variable Types 117 Modeling the responses 117 Modeling the features 118 Dealing with complex relations 119 Switching to Probabilities 121 Specifying a binary response 121 Transforming numeric estimates into probabilities 122 Guessing the Right Features 124 Defining the outcome of incompatible features 124 Solving overfitting using selection and regularization 125 Learning One Example at a Time 127 Using gradient descent 127 Understanding how SGD is different 127 Chapter 7: Introducing Neural Networks 131 Discovering the Incredible Perceptron 132 Understanding perceptron functionality 132 Touching the nonseparability limit 134 Hitting Complexity with Neural Networks 136 Considering the neuron 136 Pushing data with feed-forward 138 Going even deeper into the rabbit hole 140 Using backpropagation to adjust learning 143 Struggling with Overfitting 146 Understanding the problem 146 Opening the black box 146 Chapter 8: Building a Basic Neural Network 149 Understanding Neural Networks 150 Defining the basic architecture 151 Documenting the essential modules 153 Solving a simple problem 155 Looking Under the Hood of Neural Networks 158 Choosing the right activation function 158 Relying on a smart optimizer 160 Setting a working learning rate 161 Chapter 9: Moving to Deep Learning 163 Seeing Data Everywhere 164 Considering the effects of structure 164 Understanding Moore's implications 165 Considering what Moore's Law changes 166 Discovering the Benefits of Additional Data 167 Defining the ramifications of data 168 Considering data timeliness and quality 168 Improving Processing Speed 169 Leveraging powerful hardware 170 Making other investments 170 Explaining Deep Learning Differences from Other Forms of AI 171 Adding more layers 172 Changing the activations 174 Adding regularization by dropout 175 Finding Even Smarter Solutions 176 Using online learning 176 Transferring learning 177 Learning end to end 177 Chapter 10: Explaining Convolutional Neural Networks 179 Beginning the CNN Tour with Character Recognition 180 Understanding image basics 180 Explaining How Convolutions Work 183 Understanding convolutions 183 Simplifying the use of pooling 187 Describing the LeNet architecture 188 Detecting Edges and Shapes from Images 193 Visualizing convolutions 194 Unveiling successful architectures 196 Discussing transfer learning 197 Chapter 11: Introducing Recurrent Neural Networks 201 Introducing Recurrent Networks 202 Modeling sequences using memory 202 Recognizing and translating speech 204 Placing the correct caption on pictures 206 Explaining Long Short-Term Memory 207 Defining memory differences 208 Walking through the LSTM architecture 209 Discovering interesting variants 211 Getting the necessary attention 212 Part 3: Interacting with Deep Learning 215 Chapter 12: Performing Image Classification 217 Using Image Classification Challenges 218 Delving into ImageNet and MS COCO 219 Learning the magic of data augmentation 221 Distinguishing Traffic Signs 223 Preparing image data 224 Running a classification task 228 Chapter 13: Learning Advanced CNNs 233 Distinguishing Classification Tasks 234 Performing localization 235 Classifying multiple objects 235 Annotating multiple objects in images 237 Segmenting images 237 Perceiving Objects in Their Surroundings 239 Discovering how RetinaNet works 239 Using the Keras-RetinaNet code 241 Overcoming Adversarial Attacks on Deep Learning Applications 245 Tricking pixels 246 Hacking with stickers and other artifacts 248 Chapter 14: Working on Language Processing 251 Processing Language 252 Defining understanding as tokenization 253 Putting all the documents into a bag 254 Memorizing Sequences that Matter 257 Understanding semantics by word embeddings 257 Using AI for Sentiment Analysis 261 Chapter 15: Generating Music and Visual Art 269 Learning to Imitate Art and Life 270 Transferring an artistic style 271 Reducing the problem to statistics 272 Understanding that deep learning doesn't create 274 Mimicking an Artist 274 Defining a new piece based on a single artist 274 Combining styles to create new art 276 Visualizing how neural networks dream 276 Using a network to compose music 277 Chapter 16: Building Generative Adversarial Networks 279 Making Networks Compete 280 Finding the key in the competition 280 Achieving more realistic results 282 Considering a Growing Field 289 Inventing realistic pictures of celebrities 289 Enhancing details and image translation 290 Chapter 17: Playing with Deep Reinforcement Learning 293 Playing a Game with Neural Networks 294 Introducing reinforcement learning 294 Simulating game environments 296 Presenting Q-learning 299 Explaining Alpha-Go 302 Determining if you're going to win 303 Applying self-learning at scale 305 Part 4: The Part of Tens 307 Chapter 18: Ten Applications that Require Deep Learning 309 Restoring Color to Black-and-White Videos and Pictures 310 Approximating Person Poses in Real Time 310 Performing Real-Time Behavior Analysis 311 Translating Languages 312 Estimating Solar Savings Potential 312 Beating People at Computer Games 313 Generating Voices 314 Predicting Demographics 314 Creating Art from Real-World Pictures 315 Forecasting Natural Catastrophes 316 Chapter 19: Ten Must-Have Deep Learning Tools 317 Compiling Math Expressions Using Theano 317 Augmenting TensorFlow Using Keras 318 Dynamically Computing Graphs with Chainer 319 Creating a MATLAB-Like Environment with Torch 319 Performing Tasks Dynamically with PyTorch 320 Accelerating Deep Learning Research Using CUDA 321 Supporting Business Needs with Deeplearning4j 323 Mining Data Using Neural Designer 323 Training Algorithms Using Microsoft Cognitive Toolkit (CNTK) 324 Exploiting Full GPU Capability Using MXNet 325 Chapter 20: Ten Types of Occupations that Use Deep Learning 327 Managing People 327 Improving Medicine 328 Developing New Devices 329 Providing Customer Support 329 Seeing Data in New Ways 330 Performing Analysis Faster 331 Creating a Better Work Environment 331 Researching Obscure or Detailed Information 333 Designing Buildings 333 Enhancing Safety 334 Index 335