14 Matching Annotations
  1. Aug 2019
  2. Feb 2019
    1. Engineering Challenges
      • Communication 通常的存储都是kv,更新粒度是单个数值,但是ML算法通常的数据集类型是matrix,vector,tensor,更新的是part matrix或者vector,所以可以更进一步优化通信数据类型。

      • Fault tolerance

  3. May 2018
    1. Thesemodels are often shared globally by all worker nodes,which must frequently accesses the shared parameters asthey perform computation to refine it.

      传统模型训练流程在超大数据集上分布式训练时其模型需要所有节点都有才能训练。那么由此带来的问题:

      • 网络带宽的巨大消耗
      • 很多算法是顺序的。这种同步的训练方式很低效
      • 分布式的话容错性差
  4. Nov 2017
    1. Adversarial networks provide a strong algorithmic framework for building unsupervised learning models that incorporate properties such as common sense, and we believe that continuing to explore and push in this direction gives us a reasonable chance of succeeding in our quest to build smarter AI.
    2. This demonstration of unsupervised generative models learning object attributes like scale, rotation, position, and semantics was one of the first.
    3. Practically, this property of adversarial networks translates to better, sharper generations and higher-quality predictive models.
    4. The adversarial network learns its own cost function — its own complex rules of what is correct and what is wrong — bypassing the need to carefully design and construct one.
    5. This cost function forms the basis of what the neural network learns and how well it learns. A traditional neural network is given a cost function that is carefully constructed by a human scientist.
    6. While previous attempts to use CNNs to train generative adversarial networks were unsuccessful, when we modified their architecture to create DCGANs, we were able to visualize the filters the networks learned at each layer, thus opening up the black box.
    7. This type of optimization is difficult, and if the model weren't stable, we would not find this center point.
    8. Instead of having a neural network that takes an image and tells you whether it's a dog or an airplane, it does the reverse: It takes a bunch of numbers that describe the content and then generates an image accordingly.
    9. An adversarial network has a generator, which produces some type of data — say, an image — from some random input, and a discriminator, which gets input either from the generator or from a real data set and has to distinguish between the two — telling the real apart from the fake.
  5. May 2017
  6. Sep 2015