14 Matching Annotations
  1. Aug 2020
  2. Jun 2020
  3. May 2020
    1. WhyGeneral infrastructure simply takes time to build. You have to carefully design interfaces, write documentation and tests, and make sure that your systems will handle load. All of that is rival with experimentation, and not just because it takes time to build: it also makes the system much more rigid.Once you have lots of users with lots of use cases, it’s more difficult to change anything or to pursue radical experiments. You’ve got to make sure you don’t break things for people or else carefully communicate and manage change.Those same varied users simply consume a great deal of time day-to-day: a fault which occurs for 1% of people will present no real problem in a small prototype, but it’ll be high-priority when you have 100k users.Once this playbook becomes the primary goal, your incentives change: your goal will naturally become making the graphs go up, rather than answering fundamental questions about your system.

      The reason the conceptual architecture tends to freeze is because there is a tradeoff between a large user base and the ability to run radical experiments. If you've got a lot of users, there will always be a critical mass of complaints when the experiment blows up.

      Secondly, it takes a lot of time to scale up. This is time that you cannot spend experimenting.

      Andy here is basically advocating remaining in Explore mode a little bit longer than is usually recommended. Doing so will increase your chances of climbing the highest peak during the Exploit mode.

    2. This is obviously a powerful playbook, but it should be deployed with careful timing because it tends to freeze the conceptual architecture of the system.

      One a prototype gains some traction, conventional Silicon Valley wisdom says to scale it up. This, according to Andy Matuschak has certain disadvantages. The main drawback is that it tends to freeze the conceptual architecture of the system.

  4. Apr 2020
    1. “scaling out is the only cost-effective thing”, but plenty of successful companies managed to scale up with a handful of large machines or VMs
    2. Scaling is hard if you try do it yourself, so absolutely don’t try do it yourself. Use vendor provided, cloud abstractions like Google App Engine, Azure Web Apps or AWS Lambda with autoscaling support enabled if you can possibly avoid it.

      Scaling shall be done with cloud abstractions

  5. Mar 2020
    1. I would like to make an appeal to core developers: all design decisions involving involuntary session creation MUST be made with a great caution. In case of a high-load project, avoiding to create a session for non-authenticated users is a vital strategy with a critical influence on application performance. It doesn't really make a big difference, whether you use a database backend, or Redis, or whatever else; eventually, your load would be high enough, and scaling further would not help anymore, so that either network access to the session backend or its “INSERT” performance would become a bottleneck. In my case, it's an application with 20-25 ms response time under a 20000-30000 RPM load. Having to create a session for an each session-less request would be critical enough to decide not to upgrade Django, or to fork and rewrite the corresponding components.
  6. Jul 2019
  7. Jun 2019
    1. However, this doesn’t mean that Min-Max scaling is not useful at all! A popular application is image processing, where pixel intensities have to be normalized to fit within a certain range (i.e., 0 to 255 for the RGB color range). Also, typical neural network algorithm require data that on a 0-1 scale.

      Use min-max scaling for image processing & neural networks.

  8. Mar 2019
    1. To solve the problem of ‘scaling up’ requires ‘scaling in’ –by this we mean developing the designs and infrastructure needed to support effective use of an innovation.

      On "scaling-in" rather than "scaling-up".

  9. Sep 2018
    1. As student numbers have increased, teaching has regressed for a variety of reasons to a greater focus on information transmission and less focus on questioning, exploration of ideas, presentation of alternative viewpoints, and the development of critical or original thinking. Yet these are the very skills needed by students in a knowledge-based society.

      Related to Vijay Kumar's iron triangle. You can't increase the number of students without sacrificing quality or increasing costs.

  10. Sep 2017
  11. Aug 2017
  12. Feb 2017