Deep Learning

Deep Learning: Which Loss and Activation Functions should I use? The purpose of this post is to provide guidance on which combination of final-layer activation function and loss function should be used in a neural network depending on the business goal. This post assumes that the reader has knowledge of activation functions.  What are you trying to solve? Like all machine learning problems, the business goal determines how you should evaluate it’s success. Are you trying to predict a numerical value? Examples: Predicting the appropriate price of a product, or predicting the number of sales each day If so, s e e the section  Regression: Predicting a numerical value Are you trying to predict a categorical outcome? Examples: Predicting objects seen in an image, or predicting the topic of a conversation If so, you next need to think about how many classes there are and how many labels you wish to find. If your data is binary, it is or isn’t a class (e.g. fraud, diagnosis, likely t...

  What is the difference between cost function and activation function? A cost function is a measure of error between what value your model predicts and what the value actually is. For example, say we wish to predict the value yi for data point xi. represent the prediction or output of some arbitrary model for the point xi with parameters θ. One of the many cost functions could be this function is known as the L2 loss. Training the hypothetical model we stated above would be the process of finding the θ that minimizes this sum. -An activation function transforms the shape/representation of the data g o ing into it. A simple example could be max(0, xi), a function which outputs 0 if the input xi is negative or xi if the input xi is positive. This function is known as the “ReLU” or “Rectified Linear Unit” activation function. The choice of which function(s) are best for a specific problem using a particular neural architecture is still under a lot of discussions. However, these repre...