82 Matching Annotations
  1. Aug 2022
    1. How can you select k for k-means?  We use the elbow method to select k for k-means clustering. The idea of the elbow method is to run k-means clustering on the data set where 'k' is the number of clusters.

      Using the "elbow" or "knee of a curve" as a cutoff point is a common heuristic in mathematical optimization to choose a point where diminishing returns are no longer worth the additional cost. In clustering, this means one should choose a number of clusters so that adding another cluster doesn't give much better modeling of the data.

  2. Jul 2022
    1. Instead, werecommend wherever possible that you deploy the model itself to a server, and haveyour mobile or edge application connect to it as a web service.

      very good compatibility

    2. Once you’ve got a model you’re happy with, you need to save it so you can thencopy it over to a server where you’ll use it in production. Remember that a modelconsists of two parts: the architecture and the trained parameters. The easiest way tosave a model is to save both of these, because that way, when you load the model,you can be sure that you have the matching architecture and parameters. To saveboth parts, use the export method.

      fastai will save a file called export.pkl .pkl file is a serialised pickle file dumped using python's pickle module

    3. Many accurate models are of no use to anyone, and many inaccurate models arehighly useful. To ensure that your modeling work is useful in practice, you need toconsider how your work will be used. In 2012, Jeremy, along with Margit Zwemerand Mike Loukides, introduced a method called the Drivetrain Approach forthinking about this issue.

      Drivetrain: 传动系统。Data is power and the DL model is a drivetrain to determine the actions.

    4. Numbers that are calculated (both by linear and nonlinear layers)ParametersNumbers that are randomly initialized, and optimized (that is, the numbers thatdefine the model)


    5. At this point, we have something that is rather magical:A function that can solve any problem to any level of accuracy (the neuralnetwork) given the correct set of parametersA way to find the best set of parameters for any function (stochasticgradient descent)

      SGD may find a local optimal?

    6. universal approximation theorem

      One of the first versions of the arbitrary width case was proven by George Cybenko in 1989 for sigmoid activation functions.[8] Kurt Hornik, Maxwell Stinchcombe, and Halbert White showed in 1989 that multilayer feed-forward networks with as few as one hidden layer are universal approximators.[1] Hornik also showed in 1991[9] that it is not the specific choice of the activation function but rather the multilayer feed-forward architecture itself that gives neural networks the potential of being universal approximators.

    7. If we assign to the data attribute of a tensor, PyTorchwill not take the gradient of that step.

      Do not take the gradient of the optimisation step

      Assume you have point=torch.Tensor(size=(1,2,)); point.requires_grad_(True);

      In that case:

      point.data will be a Tensor that shares the same data with point. You can check it with: point.data_ptr(); point.data.data_ptr();

      it's is unrelated to the computation history of point.data, has requires_grad=False even point has requires_grad=True

      Any changes on initial_point.data wouldn't be tracked by autograd

    8. Rather than simplyenumerating our dataset in order for every epoch, instead what we normally do israndomly shuffle it on every epoch, before we create mini-batches.


    9. Another good reason for using mini-batches rather than calculating the gradient onindividual data items is that, in practice, we nearly always do our training on anaccelerator such as a GPU. These accelerators perform well only if they have lots ofwork to do at a time

      But not too much data.

    10. So instead we compromise: we calculate the average loss for a few data items at atime. This is called a mini-batch. The number of data items in the mini-batch iscalled the batch size. A larger batch size means that you will get a more accurate andstable estimate of your dataset’s gradients from the loss function, but it will take

      it will take longer.

    11. Let’s consider ananalogy. Imagine you are lost in the mountains with your car parked at the lowestpoint. To find your way back to it, you might wander in a random direction, but thatprobably wouldn’t help much. Since you know your vehicle is at the lowest point,you would be better off going downhill. By always taking a step in the direction ofthe steepest downward slope, you should eventually arrive at your destination.

      A good example of SGD.

    12. The gradients tell us only the slope of our function; they don’t tell us exactly how farto adjust the parameters. But they do give us some idea of how far: if the slope is

      very large which may suggest that we have more adjustments to do.

    13. As we will see, the magic of calculus allowsus to directly figure out in which direction, and by roughly how much, to changeeach weight, without having to try all these small changes. The way to do this isby calculating gradients.

      Step or Update in SGD

    14. broadcasting: it will automatically expand the tensor with the smaller rank to havethe same size as the one with the larger rank. Broadcasting is an important capabilitythat makes tensor code much easier to write.

      PyTorch use broadcasting

    15. def mnist_distance(a,b): return (a-b).abs().mean((-1,-2))mnist_distance(a_3, mean3)

      Running (valid_3_tens-mean3).abs().shape returns: torch.Size([1010, 28, 28])

      Here, if we ran (valid_3_tens-mean3).abs().mean() it would return just one value, which is not what we want. It would calculate the mean distance of every validation 3 image from the ideal 3, then get the mean of all those distances, returning one value. (Actually, try running it this way and see for yourself)

      That why we need to run (valid_3_tens-mean3).abs().mean((-1, -2)) which will return 1010 values (the remaining axis which we are not calculating the mean over) which is exactly what we want. A vector of size 1010, which is the distances of each image in the validation set from the ideal 3.

    16. It’s good to get in the habit of checking shapes as you go.

      So you can spot some early mistakes.

    17. We can also get a tensor’s rank directly with ndim:

      e.g., stacked_threes.ndim


      get the sum of all absolute difference in each pixel


  3. Jun 2022
    1. fast.ai has spent a lot of time creating cut-down versions of popular datasets that are speciallydesigned to support rapid prototyping and experimentation, and to be easier to learn with.

      indeed "fast"

    2. A window pops up containing a brief one-line explanation. The“Show in docs” link takes you to the full documentation, whereyou’ll find all the details and lots of examples. Also, most of fastai’smethods are just a handful of lines, so you can click the “source” linkto see exactly what’s going on behind the scenes.


    3. dls = TextDataLoaders.from_folder(untar_data(URLs.IMDB),valid='test')

      dls := dataLoaderS ???

    4. JARGON: TRANSFER LEARNINGUsing a pretrained model for a task different from what it was originally trainedfor.

      e.g., A physics student learns coding.

    5. fastaidefaults valid_pct to 0.2, so even if you forget, fastai will create avalidation set for you!

      valid_pct := validation (set) percentage

    6. Now execute thecell with uploader defined. It will output a button you can click, soyou can select the image you want to classify:

      Doesn't work for me.

    7. The hardest part of deep learning is artisanal: how do you know ifyou’ve got enough data, whether it is in the right format, if yourmodel is training properly, and, if it’s not, what you should do aboutit? That is why we believe in learning by doing.

      炼金术 哈哈


  4. Mar 2022
  5. Feb 2022
    1. Dennett compares consciousness to the user interface of a computer. The contents of our awareness, he asserts, bear the same relation to our brains that the little folders and other icons on the screen of a computer bear to its underlying circuitry and software. Our perceptions, memories and emotions are grossly simplified, cartoonish representations of hidden, hideously complex computations.

      Maybe conciouness is an independent part that try to figure out what the brain is doing?

    1. For example, to encourage a child who prefers chocolate candy to eat vegetables (low-frequency behavior), the behaviorist would want to make access to eating chocolate candy (high-frequency behavior) contingent upon consuming the vegetables (low-frequency behavior).

      the Premack principle

    1. Even forms of entertainment, which should be a break from the achievement mindset, incorporate it. You need to level up in the video game. You need to add more followers on Instagram. You need to improve your performance at a sport or hobby. You need to check off more items on some bucket list of films seen, books read, landmarks visited, etc.

      "The burnout society"

  6. Jan 2022
    1. We’ve seen how rebase rewrites history, while merge preserves it.

      The advantage of rebase is that it allows you to tidy up your development history.

    1. Consequently, you need to do the hard stuff first thing in the morning. The important stuff.If you don’t, it simply will not get done. By the end of your day, you’ll be exhausted. You’ll be fried. There will be a million reasons to just start tomorrow. And you will start tomorrow — which is never.So your mantra becomes: The worst comes first. Do that thing you’ve been needing to do. Then do it again tomorrow.If you take just one step toward you big goals every day, you’ll realize those goals weren’t really far away.

      So read a paper every morning?

    1. Can you do something different? I think so. Part of the answer is to spend less time consuming and more time thinking. The other part is to change your information sources from the news. Seek out dense sources of information. Some indicators you’ve found them are timeless content and direct first-hand experience. This means fewer articles and more books. If you must read the news, read it for the facts and the data, not the opinions. Let’s close with this quote by Winifred Gallagher: “Few things are as important to your quality of life as your choices about how to spend the precious resource of your free time.”

      Why you should stop reading news.

    1. Meanwhile, the people in the claims department would continue to handle the things that humans do especially well, such as complex problem solving and process innovation.


    2. With claims as with underwriting, the changes brought about by the IoT won’t mean a wholesale replacement of people by machines. Instead, the technology will do the repetitive work and the work to which it is uniquely suited.


    1. Since you're using iterm2 on a mac, another option is you can just hit CmdI, type something, and hit ESC.

      how to name tabs in Iterm2

    1. Maybe conda install --rev 1 would produce the desired behavior (restore root environment to its state after first installation).

      How to reset conda base environment

    1. It’s hard to emphasize just how important this threshold is. At less than a second to compile (and run) the tests, I can now actually continually focus on a task. Compile and logic errors are inevitable. But when I can quickly spot the error and re-compile it allows one to enter a state of flow.

      Unit Testing

    1. So as other have pointed out it is called hash, because you chop your input that you put in pieces in different places (your table entries).

      Why is it called "hash function"?

  7. Dec 2021
    1. In the strictest since, the word probability may refer to something with a great chance of happening or a small chance of happening, but in actual use, the word probability is most often used when talking about something that has a greater chance of occuring.

      In actual use, the word possibility is most often used when talking about something that has a lesser chance of occuring.

    1. The fourth and most difficult level of reading is syntopical reading. It is the most complex and systematic type of reading of all. It makes very heavy demands on the reader, even if the materials he is reading are themselves relatively easy and unsophisticated. Another name for this level might be comparative reading. When reading syntopically, the reader reads many books, not just one, and places them in relation to one another and to a subject about which they all revolve. But mere comparison of texts is not enough. … With the help of the books read, the syntopical reader is able to construct an analysis of the subject that may not be in any of the books.

      Syntopical Reading is need for a literature review.

    2. The easiest way to improve our understanding is from people who understand more about the subject than we do. So half the battle of reading for understanding is to identify and select works from someone (or a group of people) who know more about a subject than we do. The internet and Amazon have made this much easier with ratings and book reviews.

      Reading for Understanding

    1. We go by the external cues – and few people feel safe enough to display their vulnerabilities. We therefore come to assume that we are living among superior, metal-plated cyborgs rather than fragile, water-filled uncertain entities. We cannot believe that most of what we know of our own minds, especially the self-doubt, the anxiety and the sadness, must exist in those of strangers too.

      How to approach strangers.

    1. Why not use that tactic too? If you have one important thing to say or one crucial thing you need from your recipient, or one uncomfortable thing to say, try putting it in the P.S. line. This is the last impression,
      1. Tap into the power of the last impression.
    2. We all have a narcissist in us, and if you use a person’s name at ​critical moments​, you will ​increase your likelihood​ of getting an answer. For example, when you’re making a crucial request in your email, start with the recipient’s name. What’s more, research shows that ​mentioning​ the name of another person whom the recipient ​knows will also significantly raise the chances your email will be answered.
      1. Use names at critical moments
    3. 2. Add color and feeling to your email Our emails are written in ​black and white​, so they automatically look kind of boring​. Sending your thoughts in email is a bit like speaking without being able to use your body, voice, or face. So how can we put ​some color and — more importantly — feeling​ into them? By using different kinds of punctuation and, yes, ​emojis​.


    4. 1. Make an excellent first impression A subject line is your chance to make a positive first impression on your recipient. According to existing research, three things make an effective subject line: It should be short, call for action and indicate familiarity with the recipient.


    1. We can think of double loop learning as learning based on Bayesian updating — the modification of goals, rules, or ideas in response to new evidence and experience.

      then why not call it Bayesian Learning?

    1. Therefore, the distribution of predicted rewards has two bumps: one representing the possibility of falling, and one representing the possibility of successfully reaching the other side. In such situations, a standard TD algorithm learns to predict the future reward that will be received on average–in this case, failing to capture the two-peaked distribution of potential returns. A distributional reinforcement learning algorithm, on the other hand, learns to predict the full spectrum of future rewards.

      standard TD algorithm does not have the distributional information of the reward.

    2. Reinforcement learning is one of the oldest and most powerful ideas linking neuroscience and AI. In the late 1980s, computer science researchers were trying to develop algorithms that could learn how to perform complex behaviours on their own, using only rewards and punishments as a teaching signal. These rewards would serve to reinforce whatever behaviours led to their acquisition.

      e.g., the food would reinforce the behaviour that dogs salivate as soon as they hear the bell

    1. To use inline tagging, simply add the note .probability in addition to the highlight. When it's imported into Readwise, the passage will be tagged accordingly.


    1. Words work too. Often, all you need is a reason or a friendly parting line. “I have to run”; “I need to get another drink”; “do you know where the bathroom is?”; “I have to check on my friend”; “hey, it was nice talking to you”; or glancing at your phone and saying “my friend (or partner, or babysitter) is texting me,” things like that. These are reasonable needs that require you to end an interaction. Any of these things may be true, but they work as excuses too. So it’s nice to be genuine and warm about it, if you can.

      How to end a conversation without being a jerk

  8. Nov 2021
    1. Consequently, the computation for optimizing this PPO objective function is much less than that of TRPO’s. Empirically, it also proves that PPO’s performance is better than TRPO. In fact, thanks to its lightness and ease of implementation, PPO has become the default RL algorithm of OpenAI

      Essentially, PPO restricts the range that the new policy can vary from the old one.

    2. TPRO successfully addresses the problem imposed by DDPG that the performance does not improve monotonically. The subset of region lies within the constraint is called trust region. As long as the policy change is reasonably small, the approximation is not much different from the true objective function. By choosing the new policy parameters which maximizes the expectation subject to the KL divergence constraint, a lower bound of the expected long-term reward η is guaranteed. This also implies that you don’t need to worry too much about the step size with TRPO.

      Trust Region Policy Optimisation.

    1. the actor-critic architecture with two eponymous elements, actor and critic. An actor is used to tune the parameter 𝜽 for the policy function, i.e. decide the best action for a specific state.

      A critics is used for evaluating the policy estimated by the actor.

    2. Although DQN achieved huge success in higher dimensional problem, such as the Atari game, the action space is still discrete. However, many tasks of interest, especially physical control tasks, the action space is continuous.

      Policy Gradient approaches come to save.

    3. Note that these two methods require the knowledge of the transition probability p, indicating that it is a model-based algorithm.

      Two methods: Policy Iteration and Value Iteration

    4. However, model-based algorithms become impractical as the state space and action space grows (S * S * A, for a tabular setup).

      model-based RL algorithms learns the transition probability from the pair of current state s_0 and action a to the next state s_1.

    1. There is still a lot of ongoing research into MAS
      • Robotic Swarms and Planning
      • Human in the Loop System
      • Verification and Explainability
      • Learning and Decision Making
      • Agent Based Models
  9. Oct 2021
    1. Typically, RLlib collects batches of size rollout_fragment_length from rollout workers, and concatenates one or more of these batches into a batch of size train_batch_size that is the input to SGD.


    1. Location of start Runners are advised to meet outside the Hawthorns Centre where toilet facilities are also available.

      The Common park run.

    1. Its name comes from the ed command g/re/p (globally search for a regular expression and print matching lines),

      grep (Linus)

    1. It is therefore important to invest in efficient software and tools that can make top-end technologies more practical, perhaps slightly reducing performance while still providing the computation they need. 

      It's possible, human brain is so much more efficient in terms of energy use.

    1. Improving Generalization in Reinforcement Learning using Policy Similarity Embeddings

      Using behavioral similarity to measure the similarity between two states, to improve generalisation.

  10. Aug 2021
    1. a string is a sequence of characters, ie unicode codepoints; these are an abstract concept, and can't be directly stored on disk. A byte string is a sequence of, unsurprisingly, bytes - things that can be stored on disk. The mapping between them is an encoding - there are quite a lot of these (and infinitely many are possible) - and you need to know which applies in the particular case in order to do the conversion,

      string vs. byte string

    1. the @staticmethod decorator allows us to use a function without access to class instances as a class method; the @classmethod decorator can be used to call a method of a class instead of a class instance; the @property decorator is helpful when we want to access a method as an attribute.

      @staticmethod moves an outside function inside a class

    1. Two basic rules are that child classes precede parent classes and parent classes are placed in the order they were listed in.

      Dealing with the diamond problem of multi-inheritance

    1. Instead of manually writing the script name and arguments each time, you can set them in the configuration. For this in the Run area select Edit Configurations to open the Run/Debug Configurations dialog.


  11. Jul 2021
    1. Press Ctrl + v to enter into visual block mode. Use ↑ / ↓ / j / k to select multiple lines. Press Shift + i and start typing what you want. After you press Esc, the text will be inserted into all the lines you selected.


  12. Jun 2021
    1. 总的说来,就是先解放自己较为狭隘的思想,认识到世人生活得没有你想象中的那么好或坏,自己也并非一无是处,我们只需要跟自己比就够了,多专注于提升自己。然后建立自信的支撑点,也就是专注于把一件事做得比大多人都要好,从而给予自己成就感,建立起长久而真实的自信,告别无端的自卑。


    1. 总结来说就是:很累、状态不好、行动力不足但必须干活时,挑出其中最重要的一两项,先做,其他的先放着不管、当它不存在。然后把挑中的事情拆成小块小块的,告诉大脑“吃掉一块就是赢”,然后没什么压力地就“吃”起来了。“吃掉”第一块后就会触发大脑奖赏机制、收获一点成就感,因此增加了一点自身能量动力,然后能更加容易地去“吃”下一块。“吃着吃着”发现这一天也做了挺多事的,很满足。


    1. I think this has to do with https://<ip>:<port> only. When I tried http://<ip>:<port> it worked. Not sure of the reason though.

      open Jupyterlab from my local web browser

    1. Rollout means running an episode with the trained model, which you specify by passing a checkpoint directory to the command.


    1. Operating System¶ Iridis 5 is using a new version (7.4 at time of writing) of Red Hat Enterprise Linux (RHEL). This has some significant modernisations compared to RHEL 6 on Iridis 4. In particular the versions of libopenssl and glibc on RHEL 6 are showing their age and are no longer compatible with all modern codes and uses.

      That's why I cannot run Rllib on Iridis4

  13. May 2021
    1. Call numpy.full(shape, fill_value) with shape as an x, y tuple to create an array with x rows, y columns, and each element as fill_value.an_array = np.full((3, 5), 8)

      create an ndarry with all identical values

    1. If you're using bash (on a Mac or GNU/Linux distro), add this to your ~/.bashrc export PYTHONPATH="${PYTHONPATH}:/my/other/path"

      Permanently add a directory to PYTHONPATH

    1. Our vision for the AI Economist is to enable an objective study of policy impact on real-world economies, at a level of complexity that traditional economic research cannot easily address. We believe the intersection of machine learning and economics presents a wide range of exciting research directions, and gives ample opportunity for machine learning to have positive social impact.

      Still difficult to persuade people to believe it.

    1. On your Mac, open Things. Locate your repeating to-do’s template. Just start typing “repeating” and select the Quick Find list Repeating from the results. Find your template and double-click to open it. Once you’ve opened the template, you can make the following changes:

      How to edit a repeating to-do in Things3

  14. Apr 2021
    1. The same is true with my time. So I started blocking all 24 hours of my time (mental breaks, yoga workouts, prepping dinner, sleep, everything) into my digital calendar. There is no time unaccounted for. 

      This Is the Time-Management Hack That Helped Me Double My Income

    1. It's important to bear in mind that shorter code does not imply better one, so you shouldn’t misuse list comprehension when working with nested lists.

      code is better when readable and intuitive

    1. Some might argue that the power of carrots to promote accurate information will be inadequate against engagement-optimizing algorithms and human tendencies that promote misinformation.

      Exactly, why social media do that if that may reduce their traffic?