20 Matching Annotations
  1. Sep 2022
    1. To see how this plays out, we can continue looking at matrix shapes. Tracing the matrix shape through the branches and weaves of the multihead attention blocks requires three more numbers. d_k: dimensions in the embedding space used for keys and queries. 64 in the paper. d_v: dimensions in the embedding space used for values. 64 in the paper. h: the number of heads. 8 in the paper.
    1. Now, the progression of NLP, as discussed, tells a story. We begin with tokens and then build representations of these tokens. We use these representations to find similarities between tokens and embed them in a high-dimensional space. The same embeddings are also passed into sequential models that can process sequential data. Those models are used to build context and, through an ingenious way, attend to parts of the input sentence that are useful to the output sentence in translation.
  2. Apr 2022
    1. input (32x32x3)max activation: 0.5, min: -0.5max gradient: 1.08696, min: -1.53051Activations:Activation Gradients:Weights:Weight Gradients:conv (32x32x16)filter size 5x5x3, stride 1max activation: 3.75919, min: -4.48241max gradient: 0.36571, min: -0.33032parameters: 16x5x5x3+16 = 1216

      The dimensions of these first two layers are explained here

  3. Oct 2021
    1. This approach, visualizing high-dimensional representations using dimensionality reduction, is an extremely broadly applicable technique for inspecting models in deep learning.
    1. Even with this very primitive single neuron, you can achieve 90% accuracy when recognizing a handwritten text image1. To recognize all the digits from 0 to 9, you would need just ten neurons to recognize them with 92% accuracy.

      And here is a Google Colab notebook that demonstrates that

  4. Sep 2021
    1. One popular theory among machine learning researchers is the manifold hypothesis: MNIST is a low dimensional manifold, sweeping and curving through its high-dimensional embedding space. Another hypothesis, more associated with topological data analysis, is that data like MNIST consists of blobs with tentacle-like protrusions sticking out into the surrounding space.
  5. Jun 2021
  6. Feb 2021
  7. Oct 2020
  8. Sep 2020
  9. Aug 2020
  10. www.premar-atlantique.gouv.fr www.premar-atlantique.gouv.fr
    1. Article 3 : D'une largeur de 70 mètres, le chenal traversier réservé aux allers et retours entre le rivage et le large des navires et engins nautiques jouxte la zone de baignade.
  11. Jul 2020
  12. Jun 2020
  13. Apr 2020
  14. Dec 2017
    1. So thought is always both collective and individual, both a manifestation of a wider network and something unique, both an emergent property of groups and a conscious choice by some individuals to devote their scarce time and resources. The interesting questions then center on how to understand the conditions for thought. How does any society or organization make it easier for individuals to be effective vehicles for thought, to reduce the costs and increase the benefits? Or to put it in noneconomic language, how can the collective sing through the individual, and vice versa?
    2. all ideas, information, and thoughts can be seen as expressions of a collective culture that finds vehicles—people or places that are ready to provide fertile soil for thoughts to ripen. This is why such similar ideas or inventions flower in many places at the same time. It is why, too, every genius who, seen from afar, appears wholly unique looks less exceptional when seen in the dense context of their time, surrounded by others with parallel ideas and methods. Viewed in this way, it is as odd to call the individual the sole author of their ideas as it is to credit the seed for the wonders of the flowers it produces.
    3. The more dimensional any choice is, the more work is needed to think it through. If it is cognitively multidimensional, we may need many people and more disciplines to help us toward a viable solution. If it is socially dimensional, then there is no avoiding a good deal of talk, debate, and argument on the way to a solution that will be supported. And if the choice involves long feedback loops, where results come long after actions have been taken, there is the hard labor of observing what actually happens and distilling conclusions. The more dimensional the choice in these senses, the greater the investment of time and cognitive energy needed to make successful decisions.

      Conexão entre complexidade e dimensões de maiores ordem