19 Matching Annotations
  1. Aug 2019
    1. so there won’t be a blinking bunny, at least not yet, let’s train our bunny to blink on command by mixing stimuli ( the tone and the air puff)

      Is it just that how we all learn and evolve? 😲

  2. Jul 2019
    1. Compared with neural networks configured by a pure grid search,we find that random search over the same domain is able to find models that are as good or betterwithin a small fraction of the computation time.
  3. Jun 2019
    1. Throughout the past two decades, he has been conducting research in the fields of psychology of learning and hybrid neural network (in particular, applying these models to research on human skill acquisition). Specifically, he has worked on the integrated effect of "top-down" and "bottom-up" learning in human skill acquisition,[1][2] in a variety of task domains, for example, navigation tasks,[3] reasoning tasks, and implicit learning tasks.[4] This inclusion of bottom-up learning processes has been revolutionary in cognitive psychology, because most previous models of learning had focused exclusively on top-down learning (whereas human learning clearly happens in both directions). This research has culminated with the development of an integrated cognitive architecture that can be used to provide a qualitative and quantitative explanation of empirical psychological learning data. The model, CLARION, is a hybrid neural network that can be used to simulate problem solving and social interactions as well. More importantly, CLARION was the first psychological model that proposed an explanation for the "bottom-up learning" mechanisms present in human skill acquisition: His numerous papers on the subject have brought attention to this neglected area in cognitive psychology.
    1. However, this doesn’t mean that Min-Max scaling is not useful at all! A popular application is image processing, where pixel intensities have to be normalized to fit within a certain range (i.e., 0 to 255 for the RGB color range). Also, typical neural network algorithm require data that on a 0-1 scale.

      Use min-max scaling for image processing & neural networks.

  4. Mar 2019
  5. Oct 2018
    1. Do neural networks dream of semantics?

      Neural networks in visual analysis, linguistics Knowledge graph applications

      1. Data integration,
      2. Visualization
      3. Exploratory search
      4. Question answering

      Future goals: neuro-symbolic integration (symbolic reasoning and machine learning)

    1. Silvia can see patterns in her own parenting that mimic those of her mother, although she knows that her mother made many mistakes.

      Clasifico esto como neural sculpting porque lo que Silvia vio en su hogar a temprana edad (de 0-7 años) formó y determinó hasta inconscientemente el camino que ella reforzó y reprodujo cuando le tocó ser madre. Ejemplo:

  6. Aug 2017
    1. This is a very easy paper to follow, but it looks like their methodology is a simple way to improve performance on limited data. I'm curious how well this is reproduced elsewhere.

    1. The takeaway is that you should not be using smaller networks because you are afraid of overfitting. Instead, you should use as big of a neural network as your computational budget allows, and use other regularization techniques to control overfitting

      What about the rule of thumb stating that you should have roughly 5-10 times as many data points as weights in order to not overfit?

  7. Apr 2017
    1. If we write that out as equations, we get:

      It would be easier to understand what are x and y and W here if the actual numbers were used, like 784, 10, 55000, etc. In this simple example there are 3 x and 3 y, which is misleading. In reality there are 784 x elements (for each pixel) and 55,000 such x arrays and only 10 y elements (for each digit) and then 55,000 of them.

  8. Mar 2017
    1. To create it, Musk has said that he thinks we will probably have to inject a computer interface into the jugular where it will travel into the brain and unfold into a mesh of electric connections that connect directly to the neurons.

      Yeah, nothing could go wrong with this approach...

    2. Elon Musk’s neural lace project could turn us all into cyborgs, and he says that it’s only four or five years away.

      This seems incredibly ambitious--if not dangerous!

  9. Nov 2016
    1. Softmax分类器所做的就是最小化在估计分类概率(就是 Li=efyi/∑jefjLi=efyi/∑jefjL_i =e^{f_{y_i}}/\sum_je^{f_j})和“真实”分布之间的交叉熵.

      而这样的好处,就是如果样本误分的话,就会有一个非常大的梯度。而如果使用逻辑回归误分的越严重,算法收敛越慢。比如,\(t_i=1\) 而 \(y_i=0.0000001\),cost function 为 \(E=\frac{1}{2}(t-y)^2\) 那么,\(\frac{dE}{dw_i}=-(t-y)y(1-y)x_i\).

  10. Apr 2016
    1. Effect of step size. The gradient tells us the direction in which the function has the steepest rate of increase, but it does not tell us how far along this direction we should step.

      That's the reason why step size is an important factor in optimization algorithm. Too small step can cause the algorithm longer to converge. Too large step can cause that we change the parameters too much thus overstepped the optima.

  11. Jan 2016
  12. Jul 2015
  13. Jun 2015
    1. Enter the Daily Mail website, MailOnline, and CNN online. These sites display news stories with the main points of the story displayed as bullet points that are written independently of the text. “Of key importance is that these summary points are abstractive and do not simply copy sentences from the documents,” say Hermann and co.

      Someday, maybe projects like Hypothesis will help teach computers to read, too.