- Apr 2020
-
-
In the mid-90s, “COM+” (Component Services) and SOAP were popular because they reduced the risk of deploying things, by splitting them into small components
History of Microservices:
- Came from COM+ and SOAP in the mid-90s.
- Which later led to the popularisation of N-Tier ("split up the data-tier, the business-logic-tier and the presentation-tier"). This worked for some people, but the horizontal slices through a system often required changing every “tier” to finish a full change.
- Product vendors got involved and SOAP became complicated and unfashionable, which pushed people towards the "second wave" - Guerrilla SOA. This led to the proliferation of smaller, more nimble services.
- The "third wave" of SOA, branded as Microservice architecture is very popular, but often not well understood
-
all a design pattern is, is the answer to a problem that people solve so often, there’s an accepted way to solve it
“Design patterns are just bug fixes for your programming languages”
"they’re just the 1990s version of an accepted and popular stackoverflow answer"
-
Let’s do a quick run through of some very common ones:
most common design patterns:
- MVC
- ORM
- Active Record
- Repository
- Decorator
- Dependency Injection
- Factory
- Adapter
- Command
- Strategy
- Singleton
-
CDNs are web servers run by other people, all over the world. You upload your data to them, and they will replicate your data across all of their “edges” (a silly term that just means “to all the servers all over the world that they run”) so that when someone requests your content, the DNS response will return a server that’s close to them, and the time it takes them to fetch that content will be much quicker.
CDNs (Content Delivery Networks)
Offloading to a CDN is one of the easiest ways you can get extra performance for a very minimal cost
-
Understanding how a distributed cache works is remarkably simple – when an item is added, the key (the thing you use to retrieve that item) that is generated includes the address or name of the computer that’s storing that data in the cache. Generating keys on any of the computers that are part of the distributed cache cluster will result in the same key. This means that when the client libraries that interact with the cache are used, they understand which computer they must call to retrieve the data.
Distributed caching (simple explanation)
-
Memory caches are used to store the result of something that is “heavy” to calculate, takes time, or just needs to be consistent across all the different computers running your server software. In exchange for a little bit of network latency, it makes the total amount of memory available to your application the sum of all the memory available across all your servers.
Distributed caching:
- use to store results of heavy calculation
- improve consistency
- prevents "cache sampedes"
- all the major hosting providers tend to support memcached or redis compatible managed caches
-
All a load balancer does, is accept HTTP requests for your application (or from it), pick a server that isn’t very busy, and forward the request.
Load balancers:
- used when you've a lot of traffic and you're not using PaaS
- mostly operated by sysops, or are just running copies of NGINX
- you might see load balancers load balancing a "hot path" in your software onto a dedicated pool of hardware to keep it safe or isolate from failure
- you might see load balancers used to take care of SSL certificates for you (SSL Termination)
-
GraphQL gels especially well with modern JavaScript driven front ends, with excellent tools like Apollo and Apollo-Server that help optimise these calls by batching requests.
GraphQL + Apollo or Apollo-Server
-
BFF is an API that serves one, and specifically only one application
BFF (backend for frontend):
- translates an internal domain into the terminal language of the app it serves
- takes care of authentication, rate limiting, and other stuff you don't want to do more than once
- reduces needless roundtrips to the server
- translates data to be more suitable for its target app
- is pretty front-end-dev friendly. Queries and schema strongly resemble JSON & it keeps your stack "JS all the way down" without being beholden to some distant backend team
-
GraphQL really is just a smart and effective way to schema your APIs, and provide a BFF – that’s backend for frontend
-
What sets GraphQL apart a little from previous approaches (notably Microsofts’ OData) is the idea that Types and Queries are implemented with Resolver code on the server side, rather than just mapping directly to some SQL storage.
What makes GraphQL favorable over other implementations.
In result:
- GraphQL can be a single API over a bunch of disparate APIs in your domain
- it solves the "over fetching" problem that's quite common in REST APIs (by allowing the client to specify a subset of data to return)
- acts as an anti-corruption layer of sorts, preventing unbounded access to underlying storage
- GraphQL is designed to be the single point of connection that your web or mobile app talks to (it highly optimises performance)
-
GraphQL is confusingly, a Query Language, a standard for HTTP APIs and a Schema tool all at once.
GraphQL
-
Use a swagger or OpenAPI library to generate a schema and you’re pretty much doing what most people are doing.
Generating RESTful schema
-
JSON-RPC is “level 0” of the Richardson Maturity Model – a model that describes the qualities of a REST design.
JSON-RPC.
You can build it by:
- using HTTP VERBs correctly
- organising your APIs into logical "resources", e.g. ("customer", "product", "catalogue")
- using correct HTTP response codes for interactions with your API
-
REST is a good architectural style (and it is, a lot of the modern naysaying about REST is relatively uninformed and not too dissimilar to the treatment SOAP had before it)
REST is still a good architectural style
-
there was a push for standardisation of “SOAP” (simple object access protocol) APIs
Standarisation of SOAP brought a lot of good stuff but people found XML cumbersome to read.
A lot of things being solved in SOAP had to subsequently be re-solved on top of JSON using emerging open-ish standards like Swagger (now OpenAPI) and JSON:API
-
The basics of HTTP are easy to grasp – there’s a mandatory “request line”
Mandatory HTTP request line:
- verb (GET, POST, PUT and HEAD most frequently)
- URL (web address)
- protocol version (HTTP/1.1)
Then, there's a bunch of optional request header fields.
Example HTTP request:
GET http://www.davidwhitney.co.uk/ HTTP/1.1 Host: www.davidwhitney.co.uk Connection: keep-alive User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64… Accept: text/html,application/xhtml+xml,application/xml;q=0.9 Accept-Encoding: gzip, deflate Accept-Language: en-GB,en-US;q=0.9,en;q=0.8Example response:
HTTP/1.1 200 OK Cache-Control: public,max-age=1 Content-Type: text/html; charset=utf-8 Vary: Accept-Encoding Server: Kestrel X-Powered-By: ASP.NET Date: Wed, 11 Dec 2019 21:52:23 GMT Content-Length: 8479 <!DOCTYPE html> <html lang="en">... -
web is an implementation of the design pattern REST – which stands for “Representational State Transfer”. You’ll hear people talk about REST a lot – it was originally defined by Roy Fielding in his PhD dissertation, but more importantly was a description of the way HTTP/1.0 worked at the time
Origins of the REST design pattern
-
So the web is RESTful by default. REST describes the way HTTP works.
-
Most APIs you use, or build will be “REST-ish”.
- You'll be issuing the same kind of "HTTP requests" as the browser
- Mostly you'll get JSON responses (sometimes XML)
- You can describe these APIs as JSON-RPC or XML-RPC
-
Honestly for the most part it’s a matter of taste, and they’re all perfectly appropriate ways to build web applications.
Which model to choose:
- Server-rendered MVC - good for low-interactivity websites. Web apps on the other side have their complexity cost
- SPAs (React, Angular, Vue) offer high fidelity UX. The programming models work well for responsive UX
- Static sites - great for blogs, marketing microsites, CMS (anything where content is the most important). They scale well, basically cannot crash, and are cheap to run
-
static site generators became increasingly popular – normally allowing you to use your normal front-end web dev stack, but then generating all the files using a build tool to bundle and distribute to dumb web servers or CDNs
Examples of static site generators: Gatsby, Hugo, Jekyll, Wyam
-
MVVM is equally common in single page apps where there’s two way bindings between something that provides data (the model) and the UI (which the view model serves).
-
SPAs are incredibly common, popularised by client-side web frameworks like Angular, React and Vue.js.
SPAs:
- popularised by client-side web frameworks like Angular, React and Vue.js
- real difference between the MVP app is shifting most of its work on the client side
- there's client side MVC, MVVM (model-view-view-model) and FRP (functional reactive programming)
Angular - client side MVC framework following its pattern, except it's running inside the users web browser.
React - implementation of FRP. A little more flexible, but more concerned with state change events in data (often using some event store like Redux)
-
http://www.mycoolsite.com/Home/Index
With such a website, MVC model would try to:
- find a "
HomeController" file/module (depending on programming language) inside the controllers directory. - "
Index" function would probably exist and would return a model (some data) that would be rendered by a view (HTML template from the views folder)
(All the different frameworks do this slightly differently, but the core idea stays the same – features grouped together by controllers, with functions for returning pages of data and handling input from the web)
- find a "
-
when most people say “MVC” they’re really describing “Rails-style” MVC apps where your code is organised into a few different directories
Different directories of the MVC pattern:
- /controllers
- /models
- /views
-
most people stick to a handful of common patterns
Common patterns:
- The MVC App
- The Single Page App with an API
- Static Sites Hosted on a CDN or other dumb server
- Something else...
-
There are tonne of general purpose web servers out there
Realistically, you'll see a mixture of:
- Apache
- NGINX
- Microsoft IIS
- along with some development stack specific web servers (Node.js, ASP.NET CORE for C#, HTTP4K for Kotlin)
-
“web servers”, implement the “HTTP protocol” – a series of commands you can send to remote computers – that let you say “hey, computer, send me that document”
Web servers (simple explanation)
-
-
www.dobrarekruterka.pl www.dobrarekruterka.pl
-
He is intrigued by the unique qualities of each person, organizes for maximum productivity, has a great desire to learn, can sense other people’s feelings, and is introspective and appreciates intellectual discussions—strengths that have been confirmed both through the Clifton StrengthsFinder assessment and are reflected in his personal and professional life.
Way in which you can summarise the results of Gallup's test on your LinkedIn.
Tags
Annotators
URL
-
-
podcastnotes.org podcastnotes.org
-
People pay for premium content for the content itself and connection to the creator, not for studio lighting and animations
-
-
stats.stackexchange.com stats.stackexchange.com
-
Repeated measures involves measuring the same cases multiple times. So, if you measured the chips, then did something to them, then measured them again, etc it would be repeated measures. Replication involves running the same study on different subjects but identical conditions. So, if you did the study on n chips, then did it again on another n chips that would be replication.
Difference between repeated measures and replication
-
-
rakyll.medium.com rakyll.medium.com
-
I’m sharing a few insights I specifically found useful for developers who are not specialized in this domain.
Insights on databases from a Google engineer:
- You are lucky if 99.999% of the time network is not a problem.
- ACID has many meanings.
- Each database has different consistency and isolation capabilities.
- Optimistic locking is an option when you can’t hold a lock.
- There are anomalies other than dirty reads and data loss.
- My database and I don’t always agree on ordering.
- Application-level sharding can live outside the application.
- AUTOINCREMENT’ing can be harmful.
- Stale data can be useful and lock-free.
- Clock skews happen between any clock sources.
- Latency has many meanings.
- Evaluate performance requirements per transaction.
- Nested transactions can be harmful.
- Transactions shouldn’t maintain application state.
- Query planners can tell a lot about databases.
- Online migrations are complex but possible.
- Significant database growth introduces unpredictability.
-
-
simonwillison.net simonwillison.net
-
So I wrote a GitHub Actions workflow that automatically updates the README page in the repo every time a new TIL markdown file is added or updated!
You can use GitHub Actions to automatically rewrite your README.md
Tags
Annotators
URL
-
-
news.ycombinator.com news.ycombinator.com
-
Think of mental laziness as a lack of mental exercise. Mental exercise, such as tough decisions, actually burn more calories and impact your overall physiology.
Mental laziness
-
Laziness is stagnation without regard for enjoyment or perceptions. In other words it is the opposite of ambition or progression.
Laziness - interesting definition
-
-
-
At the company I work at, one of our products is an embeddable commenting system. Unlike single-page applications, when we encounter bugs they’re usually on the client’s website. This raised the question, how can we embed a piece of code that will run on all our client’s websites, that will help us debug and improve our overall build experience.
Case when userscripts apply (not extensions)
Tags
Annotators
URL
-
-
byorgey.wordpress.com byorgey.wordpress.com
-
“Hey, I have a good idea for a game,” I said. “It’s called the function machine game. I will think of a function machine. You tell me things to put into the function machine, and I will tell you what comes out. Then you have to guess what the function machine does.” He immediately liked this game and it has been a huge hit; he wants to play it all the time. We played it while driving to a party yesterday, and we played it this morning while I was in the shower.
Great idea for a game with your kids to develop logical thinking in them
-
-
skutecznyprogramista.pl skutecznyprogramista.pl
-
Z gier można wyciągnąć też inną naukę. Jeśli Twoim celem jest przejście do następnej lokacji, to czy musisz wykonywać wszystkie zadania poboczne?No nie musisz. Dlatego wyżej, gdy podawałem wymagane umiejętności dla osoby, która prowadzi szkolenia z Reacta, albo pracuje dla startupów, to napisałem “dobra znajomość JSa”, bo “doskonała” nie pomoże Ci w osiągnięciu tego celu.
Don't overlearn
-
W miarę jak będziesz się rozwijać i zdobywać nowe informacje, Twój plan będzie trzeba dostosować do nowych warunków.
Be prepared for a change. Be flexible in defining your goals
-
Arnie miał wielkie plany, ale nie realizował wszystkich na raz.Skupił się na jednej rzeczy - kulturystyce - bo wiedział, że to otworzy mu drogę do Ameryki i do aktorstwa.
Think BIG, act small
(small actions lead to big changes)
-
Jeśli wybierzesz kilka rzeczy na raz to ryzykujesz, że znowu zaczniesz miotać się we wszystkich kierunkach.Nie polecam takiej opcji, bo właśnie przez takie myślenie kończymy potem wkurwieni, z siwymi włosami i podkrążonymi oczami.Doświadczyłem wszystkich tych trzech objawów i dopiero kiedy skupiłem się na jednej rzeczy, to odzyskałem balans i przestałem się denerwować.
It's not so effective to have dozens of goals, but the one you can purely focus on
-
Niestety, jeśli faktycznie chcesz być najlepszą osobą, to z moich obserwacji wynika, że trzeba zapierdzielać.
Wanna be the best in what you're doing? Oh boy, be prepared...
-
W czasach młodości Arniego, Reg Park był wielką gwiazdą kulturystyki, sławnym oraz bogatym aktorem i otaczały go piękne dziewczyny.Arnie chciał się wydostać z zadupia w Austrii, przeprowadzić do Ameryki i mieć dokładnie to samo co Reg.Wolność, sławę, kasę i dziewczyny.Wszystkie jego działania były podporządkowane dotarciu do tego celu.Arnie dokładnie zawęził czego chce.
Taking example from Arnold Schwarzenegger we shall have clear & precise goals
-
👉 Wiele osób ma cele w stylu “chcę się nauczyć Reacta”. I jest to zbyt słabo sprecyzowany cel.👉 Celem musi być coś w stylu “chcę się nauczyć Reacta, żeby pracować w firmie X i robić projekty dla startupów z Doliny Krzemowej”.👉 Albo “chcę się nauczyć Reacta, żeby pracować w Facebooku z Danem Abramovem nad kolejnymi wersjami Reacta”.👉 Albo “chcę się nauczyć Reacta, żeby prowadzić szkolenia stacjonarne dla backendowców, którzy nie umieją we frontendy”.
Instead of planning to "learn machine learning" give it a bit more details, e.g.:
- "I want to learn machine learning to work at Amazon with the latest technologies"
In result, achieving your goal will become different (imho better)
-
Pytać wszystkich wszędzie i powiększaj swoją znajomość tematu. Poszerzaj worek możliwości.Przeszukaj Reddita, Stacka, grupy na Facebooku. Znajdź jakiegoś mądrego człowieka na social media i napisz mu DM.
Ask everyone/everywhere for advice if you really don't know what you want to do
-
-
tauday.com tauday.com
-
Tau Day is an annual celebration of the circle constant τ=6.283185…\tau = 6.283185\ldots, which takes place every June 28 (6/28 in the American calendar system).
Tau (τ) = 2π radians = 360°
It's simply a more intuitive way of representing a full circular rotation. Possibly Euler could've changed it 100 years ago, but too many textbooks already applied the concept of 2π radians instead
-
-
code.visualstudio.com code.visualstudio.com
-
To customize settings for debugging tests, you can specify "request":"test" in the launch.json file in the .vscode folder from your workspace.
Customising settings for debugging tests while running
Python: Debug All Tests
or
Python: Debug Test Method
-
For example, the test_decrement functions given earlier are failing because the assertion itself is faulty.
Debugging tests themselves
- Set a breakpoint on the first line of the failing function (e.g.
test_decrement) - Click the "Debug Test" option above the function
- Open Debug Console and type:
inc_dec.decrement(3)to see what is the actual output when we use x=3 - Stop the debugger and correct the tests
- Save the test file and run the tests again to look for a positive result
- Set a breakpoint on the first line of the failing function (e.g.
-
Support for running tests in parallel with pytest is available through the pytest-xdist package.
pytest-xdist provides support for parallel testing.
- To enable it on Windows:
py -3 -m pip install pytest-xdist- Create a file
pytest.iniin your project directory and specify in it the number of CPUs to be used (e.g. 4):[pytest] addopts=-n4 - Run your tests
-
With pytest, failed tests also appear in the Problems panel, where you can double-click on an issue to navigate directly to the test
pytest displays failed tests also in PROBLEMS
-
VS Code also shows test results in the Python Test Log output panel (use the View > Output menu command to show the Output panel, then select Python Test Log
Another way to view the test results:
View > Output > Python Test Log
-
For Python, test discovery also activates the Test Explorer with an icon on the VS Code activity bar. The Test Explorer helps you visualize, navigate, and run tests
Test Explorer is activated after discovering tests in Python:
-
Once VS Code recognizes tests, it provides several ways to run those tests
After discovering tests, we can run them, for example, using CodeLens:
-
You can trigger test discovery at any time using the Python: Discover Tests command.
After using
python.testing.autoTestDiscoverOnSaveEnabled, it'll be set totrueand discovering tests whenever a test file is saved.If discovery succeeds, the status bar shows Run Tests instead:
-
Sometimes tests placed in subfolders aren't discovered because such test files cannot be imported. To make them importable, create an empty file named __init__.py in that folder.
Tip to use when tests aren't discoverable in subfolderds (create
__init__.pyfile) -
Testing in Python is disabled by default. To enable testing, use the Python: Configure Tests command on the Command Palette.
Start testing in VS Code by using
Python: Configure Tests(it automatically chooses one testing framework and disables the rest).Otherwise, you can configure tests manually by setting only one of the following to True:
python.testing.unittestEnabledpython.testing.pytestEnabledpython.testing.nosetestsEnabled
-
python.testing.pytestArgs: Looks for any Python (.py) file whose name begins with "test_" or ends with "_test", located anywhere within the current folder and all subfolders.
Default behaviour of test discovery by pytest framework
-
python.testing.unittestArgs: Looks for any Python (.py) file with "test" in the name in the top-level project folder.
Default behaviour of test discovery by unittest framework
-
Create a file named test_unittest.py that contains a test class with two test methods
Sample test file using unittest framework.
inc_decis the file that's being tested:import inc_dec # The code to test import unittest # The test framework class Test_TestIncrementDecrement(unittest.TestCase): def test_increment(self): self.assertEqual(inc_dec.increment(3), 4) # checks if the results is 4 when x = 3 def test_decrement(self): self.assertEqual(inc_dec.decrement(3), 4) if __name__ == '__main__': unittest.main() -
Each test framework has its own conventions for naming test files and structuring the tests within, as described in the following sections. Each case includes two test methods, one of which is intentionally set to fail for the purposes of demonstration.
- each testing framework has own naming conventions
- each case includes two test methods (one of which fails)
-
Nose2, the successor to Nose, is just unittest with plugins
Nose2 testing
-
Python tests are Python classes that reside in separate files from the code being tested.
-
general background on unit testing, see Unit Testing on Wikipedia. For a variety of useful unit test examples, see https://github.com/gwtw/py-sorting
-
Running the unit test early and often means that you quickly catch regressions, which are unexpected changes in the behavior of code that previously passed all its unit tests.
Regressions
-
Developers typically run unit tests even before committing code to a repository; gated check-in systems can also run unit tests before merging a commit.
When to run unit tests:
- before committing
- ideally before merging
- many CI systems run it after every build
-
in unit testing you avoid external dependencies and use mock data or otherwise simulated inputs
Unit tests are small, isolated piece of code making them quick and inexpensive to run
-
The practice of test-driven development is where you actually write the tests first, then write the code to pass more and more tests until all of them pass.
Essence of TDD
-
The combined results of all the tests is your test report
test report
-
each test is very simple: invoke the function with an argument and assert the expected return value.
e.g. test of an exact number entry:
def test_validator_valid_string(): # The exact assertion call depends on the framework as well assert(validate_account_number_format("1234567890"), true) -
Unit tests are concerned only with the unit's interface—its arguments and return values—not with its implementation
-
unit is a specific piece of code to be tested, such as a function or a class. Unit tests are then other pieces of code that specifically exercise the code unit with a full range of different inputs, including boundary and edge cases.
Essence of unit testing
-
-
-
Open Command tool and search for coverage
Checking CSS coverage inside Chrome Dev Tools:
- [F12] (open up dev tools)
- Click the 3 dots next to "x"
- "More tools" > "Coverage"
-
-
-
Stroną, która jako jedyna wyświetla wszystkie dostępne do śledzenia statki powietrze jest ADSBexchange.com, którą zasilimy naszymi danymi. Jest to największa na świecie społeczność właścicieli odbiorników ADS-B / Mode S / MLAT i jednocześnie największe na świecie, w pełni publiczne i darmowe, źródło niefiltrowanych danych o lotach. Zapewniany przez to narzędzie dostęp do danych lotów na całym świecie jest wykorzystywany zarówno przez hobbystów, badaczy, jak i, co ciekawe, dziennikarzy.
ADSBexchange.com in comparison to commercial sites like FlightAware or Flightradar24 doesn't hide any flights (as here no one can pay to hide them from the public access).
-
-
www.goldyarora.com www.goldyarora.com
-
Can G Suite Admin Read My Email?
tl;dr: YES (check the article for detailed steps)
-
-
github.com github.com
-
AinD launches Android apps in Docker, by nesting Anbox containers inside Docker.
AinD - useful tool when we need to run an Android app 24/7 in the cloud.
Unlike the alternatives, AinD is not VM, but IaaS based
Tags
Annotators
URL
-
-
johnpapa.net johnpapa.net
-
gh repo create hello-world -d "A react app for the web" --public
GitHub released a new CLI: gh with which you can do much more operations.
For example, you can create repo without going into your browser:
gh repo create hello-world -d "A react app for the web" --publicGenerally, it will be great for CI/CD pipelines
Tags
Annotators
URL
-
-
-
An emerging way to bypass the need for passwords is to use magic links. A magic link is a temporary URL that expires after use, or after a specific interval of time. Magic links can be sent to your email address, an app, or a security device. Clicking the link authorizes you to sign in.
Magic Links to replace passwords?
-
Hashing passwords won’t save you or your users. Once a database of passwords has been stolen, hackers aim immense distributed computing power at those password databases. They use parallel GPUs or giant botnets with hundreds of thousands of nodes to try hundreds of billions of password combinations per second in hopes of recovering plaintext username/password pairs.
What happens when the database is in hacker's hands
-
-
medium.com medium.com
-
The solution will be to go for serverless functions. This means that instead of occupying a server completely, it will only use the server capacity when the function needs to run.
Serverless as a cheap back end option
-
My favourites are Zeit and Netlify. They are quite similar in the features they provide: continuous deployment, around 100GB of bandwidth per month, and a built-in CDN. Another benefit is that they both provide the option of serverless functions, as we will see in the next section. This simplifies the number of services we need to integrate for our entire stack.
Good website hosting:
Zeit or Netlify
(CD + 100 GB of bandwith / month + built-in CDN)
-
-
en.wikipedia.org en.wikipedia.org
-
Guédelon Castle (Château de Guédelon) is a castle currently under construction near Treigny, France. The castle is the focus of an experimental archaeology project aimed at recreating a 13th-century castle and its environment using period technique, dress, and material.
Guédelon Castle (Château de Guédelon)
More info on HN
Tags
Annotators
URL
-
-
github.com github.com
-
To take full advantage of tabular augmentation for time-series you would perform the techniques in the following order: (1) transforming, (2) interacting, (3) mapping, (4) extracting, and (5) synthesising
-
process of modular feature engineering and observation engineering while emphasising the order of augmentation to achieve the best predicted outcome from a given information set
Tabular Data Augmentation
Tags
Annotators
URL
-
-
news.ycombinator.com news.ycombinator.com
-
1) Redash and Falcon focus on people that want to do visualizations on top of SQL2) Superset, Tableau and PowerBI focus on people that want to do visualizations with a UI3) Metabase and SeekTable focus on people that want to do quick analysis (they are the closest to an Excel replacement)
Comparison of data analysis tools:
1) Redash & Falcon - SQL focus
2) Superset, Tableau & PowerBI - UI workflow
3) Metabase & SeekTable - Excel like experience
Tags
Annotators
URL
-
-
code.visualstudio.com code.visualstudio.com
-
Visual Studio Code supports working with Jupyter Notebooks natively, as well as through Python code files.
To run cells inside a Python script in VSCode, all you need to is to define Jupyter-like code cells within Python code using a
# %%comment:# %% msg = "Hello World" print(msg) # %% msg = "Hello again" print(msg)
-
-
blog.jupyter.org blog.jupyter.org
-
JupyterLab project, which enables a richer UI including a file browser, text editors, consoles, notebooks, and a rich layout system.
How JupyterLab differs from a traditional notebook
-
-
rachelbythebay.com rachelbythebay.com
-
I could probably bootstrap my way up from this with the C compiler to write a terrible editor, then write a terrible TCP client, find my way out to ftp.gnu.org, get wget, and keep going from there. Assume that documentation is plentiful. You want a copy of the Stevens book so you can figure out how to do a DNS query by banging UDP over the network? Done.
What would the author do in a situation of being alone in a room with:
HD #1 is blank. HD #2 has a few scraps of a (Linux) OS on it: bootloader, kernel, C library and compiler, that sort of thing. There's a network connection of some sort, and that's about it. There are no editors and nothing more advanced than 'cat' to read files. You don't have jed, joe, emacs, pico, vi, or ed (eat flaming death). Don't even think about X. telnet, nc, ftp, ncftp, lftp, wget, curl, lynx, links? Luxury! Gone. Perl, Python and Ruby? Nope.
Tags
Annotators
URL
-
-
twitter.com twitter.com
-
B&W is compelling because by removing color detail it draws attention to figure, curves, facial expressions, emotions, movement, time of day, depending on subject
What B&W photos expose
-
-
marginalrevolution.com marginalrevolution.com
-
We find three important trends in the evolution of musical discourse: the restriction of pitch sequences (with metrics showing less variety in pitch progressions), the homogenization of the timbral palette (with frequent timbres becoming more frequent), and growing average loudness levels. The picture at right shows the timbral variety: Smaller values of β indicate less timbral variety: frequent codewords become more frequent, and infrequent ones become even less frequent. This evidences a growing homogenization of the global timbral palette. It also points towards a progressive tendency to follow more fashionable, mainstream sonorities.
Statistical evidence that modern pop music is boring or at least more homogeneous than in the past.
Dataset used: Million Song Dataset (which doesn't contain entries from the most recent years)
-
-
-
Unfortunately no - it cannot be done without Trusted security devices. There are several reasons for this. All of the below is working on the assumption you have no TPM or other trusted security device in place and are working in a "password only" environment.
Devices without a TPM module will be always asked for a password (e.g. by BitLocker) on every boot
-
-
medium.com medium.com
-
Most people are surprised to learn that of Plotly’s 50 engineers, the vast majority are React developers. This is because Dash is primarily a frontend library — there are far more lines of JavaScript (Typescript) than Python, R, or Julia code. Plotly only has 3 full-time Python developers, 2 full-time R developers, and 0 full-time Julia developers.
Who works behind Plotly/Dash: 50 engineers:
- 45 JavaScript (?)
- 3 Python
- 2 R
- 0 Julia
-
Behind the scenes, when a Python, R or Julia engineer creates a Dash app, they are actually creating a React Single Page Application (“SPA”).
-
With Dash, any open-source React UI component can be pulled from npm or GitHub, stirred with water, transmogrified into a Dash component, then imported into your Dash app as a Python, R, or Julia library. C’est magnifique! 👨🍳 Dash makes the richness and innovation of the React frontend ecosystem available to Python, R, and Julia engineers for the first time.
Dash components are based on React
-
-
code.visualstudio.com code.visualstudio.com
-
To use Gunicorn as your web server, it must be included in the requirements.txt file as an app dependency. It does not need to be installed in your virtual environment/host machine.
-
-
stackoverflow.com stackoverflow.com
-
pip is a package manager. conda is both a package manager and an environment maanger.
conda vs pip:
-
-
code.visualstudio.com code.visualstudio.com
-
To help you get started quickly, we created a special Installer of Visual Studio Code for Java developers. Download Visual Studio Code Java Pack Installer Note: The installer is currently only available for Windows. For other OS, please install those components (JDK, VS Code and Java extensions) individually. We're working on the macOS version, please stay tuned. The package can be used as a clean install or an update for an existing development environment to add Java or Visual Studio Code. Once downloaded and opened, it automatically detects if you have the fundamental components in your local development environment, including the JDK, Visual Studio Code, and essential Java extensions.
If you wish to use Java inside VSCode, try downloading the Installer of Visual Studio Code for Java developers
Tags
Annotators
URL
-
-
code.visualstudio.com code.visualstudio.com
-
Note: When you create a new virtual environment, you should be prompted by VS Code to set it as the default for your workspace folder. If selected, the environment will automatically be activated when you open a new terminal.
After creating a new project related environment, it shall be specified as a default for this specific project
-
Tip: Use Logpoints instead of print statements: Developers often litter source code with print statements to quickly inspect variables without necessarily stepping through each line of code in a debugger. In VS Code, you can instead use Logpoints. A Logpoint is like a breakpoint except that it logs a message to the console and doesn't stop the program. For more information, see Logpoints in the main VS Code debugging article.
Try to use logpoints instead of print statements.
More info: https://code.visualstudio.com/docs/editor/debugging#_logpoints
-
-
code.visualstudio.com code.visualstudio.com
-
Open an Anaconda command prompt and run conda create -n myenv python=3.7 pandas jupyter seaborn scikit-learn keras tensorflow
Command to quickly create a new Anaconda environment:
conda create -n myenv python=3.7 pandas jupyter seaborn scikit-learn keras tensorflow
-
-
dfrieds.com dfrieds.com
-
On the job, if you notice poor infrastructure, speak up to your manager early on. Clearly document the problem, and try to incorporate a data engineering, infrastructure, or devops team to help resolve the issue! I'd also encourage you to learn these skills too!
Recommendation to Data Scientists dealing with poor infrastructure
-
When I worked at Target HQ in 2012, employees would arrive to work early - often around 7am - in order to query the database at a time when few others were doing so. They hoped they’d get database results quicker. Yet, they’d still often wait several hours just to get results.
What poor data infrastructure leads to
-
The Data Scientist, in many cases, should be called the Data Janitor. ¯_(ツ)_/¯
:D
-
In regards to quality of data on the job, I’d often compare it to a garbage bag that ripped, had its content spewed all over the ground and your partner has asked you to find a beautiful earring that was accidentally inside.
From my experience, I can only agree
-
On the job, I’d recommend you document your work well and calculate the monetary value of your analyses based on factors like employee salary, capital investments, opportunity cost, etc. These analyses will come in handy for a promotion/review packet later too.
Factors to keep a track of as a Data Scientist
-
you as a Data Scientist should try to find a situation to be incredibly valuable on the job! It’s tough to find that from the outskirts of applying to jobs, but internally, you can make inroads supporting stakeholders with evidence for their decisions!
Try showcasing the evidence of why the employer needs you
-
As a Data Scientist in the org, are you essential to the business? Probably not. The business could go on for a while and survive without you. Sales will still be made, features will still get built, customer support will handle customer concerns, etc.
As a Data Scientist, you are more of a "support" to the overall team
-
As the resident Data Scientist, you may become easily inundated with requests from multiple teams at once. Be prepared to ask these teams to qualify and defend their requests, and be prepared to say “no” if their needs fall outside the scope of your actual priority queue. I’d recommend utilizing the RICE prioritization technique for projects.
Being the only data person be sure to prioritise the requests
-
Common questions are: How many users click this button; what % of users that visit a screen click this button; how many users have signed up by region or account type? However, the data needed to answer those questions may not exist! If the data does exists, it’s likely “dirty” - undocumented, tough to find or could be factually inaccurate. It’ll be tough to work with! You could spend hours or days attempting to answer a single question only to discover that you can’t sufficiently answer it for a stakeholder. In machine learning, you may be asked to optimize some process or experience for consumers. However, there’s uncertainty with how much, if at all, the experience can be improved!
Common types of problems you might be working with in Data Science / Machine Learning industry
-
In one experience, a fellow researcher spent over a month researching a particular value among our customers through qualitative and quantitative data. She presented a well-written and evidence-backed report. Yet, a few days later, a key head of product outlined a vision for the team and supported it with a claim that was antithetical to the researcher’s findings! Even if a data science project you advocate for is greenlighted, you may be on your own as the rare knowledgeable person to plan and execute it. It’s unlikely leadership will be hands-on to help you research and plan out the project.
Data science leadership is sorely lacking
-
Because people don’t know what data science does, you may have to support yourself with work in devops, software engineering, data engineering, etc.
You might have multiple roles due to lack of clear "data science" terminology
-
In 50+ interviews for data related jobs, I’ve been asked about AB testing, SQL analytics questions, optimizing SQL queries, how to code a game in Python, Logistic Regression, Gradient Boosted Trees, data structures and/or algorithms programming problems!
Data science interviews lack a clarity of scope
-
seven most common (and at times flagrant) ways that data science has failed to meet expectations in industry
- People don’t know what “data science” does.
- Data science leadership is sorely lacking.
- Data science can’t always be built to specs.
- You’re likely the only “data person.”
- Your impact is tough to measure — data doesn’t always translate to value.
- Data & infrastructure have serious quality problems.
- Data work can be profoundly unethical. Moral courage required.
-
-
datascience.stackexchange.com datascience.stackexchange.com
-
In data science community the performance of the model on the test dataset is one of the most important things people look at. Just look at the competitions on kaggle.com. They are extremely focused on test dataset and the performance of these models is really good.
In data science, performance of the model on the test model is the most important metric for the majority.
It's not always the best measurement since the most efficient model can completely misperform while receiving a different type of a dataset
-
It's basically a look up table, interpolating between known data points. Except, unlike other interpolants like 'linear', 'nearest neighbour' or 'cubic', the underlying functional form is determined to best represent the kind of data you have.
You can describe AI/ML methods as a look up table that adjusts to your data points unlike other interpolants (linear,nearest neighbor or cubic)
-
-
www.quora.com www.quora.com
-
there must be a space before any punctuation sign that is made of two parts ( ; : ! ? ) and no space before any other punctuation sign
French rule
-
-
-
git reflog is a very useful command in order to show a log of all the actions that have been taken! This includes merges, resets, reverts: basically any alteration to your branch.
Reflog - shows the history of actions in the repo.
With this information, you can easily undo changes that have been made to a repository with
git resetgit reflogSay that we actually didn't want to merge the origin branch. When we execute the git reflog command, we see that the state of the repo before the merge is at
HEAD@{1}. Let's perform a git reset to point HEAD back to where it was onHEAD@{1}! -
it pull is actually two commands in one: a git fetch, and a git merge. When we're pulling changes from the origin, we're first fetching all the data like we did with a git fetch, after which the latest changes are automatically merged into the local branch.
Pulling - downloads content from a remote branch/repository like
git fetchwould do, and automaticallymerges the new changesgit pull origin master -
It doesn't affect your local branch in any way: a fetch simply downloads new data.
Fetching - downloads content from a remote branch or repository without modifying the local state
git fetch origin master -
When a certain branch contains a commit that introduced changes we need on our active branch, we can cherry-pick that command! By cherry-picking a commit, we create a new commit on our active branch that contains the changes that were introduced by the cherry-picked commit.
Cherry-picking - creates a new commit with the changes that the cherry-picked commit introduced.
By default, Git will only apply the changes if the current branch does not have these changes in order to prevent an empty commit
git cherry-pick 76d12 -
Another way of undoing changes is by performing a git revert. By reverting a certain commit, we create a new commit that contains the reverted changes!
Reverting - reverts the changes that commits introduce. Creates a new commit with the reverted changes
git revert ec5be -
Sometimes, we don't want to keep the changes that were introduced by certain commits. Unlike a soft reset, we shouldn't need to have access to them any more.
Hard reset - points
HEADto the specified commit.Discards changes that have been made since the new commit that
HEADpoints to, and deletes changes in working directorygit reset --hard HEAD~2 git status -
git rebase copies the commits from the current branch, and puts these copied commits on top of the specified branch.
Rebasing - copies commits on top of another branch without creating a commit, which keeps a linear history.
Changes the history as new hashes are created for the copied commits.
git rebase masterA big difference compared to merging, is that Git won't try to find out which files to keep and not keep. The branch that we're rebasing always has the latest changes that we want to keep! You won't run into any merging conflicts this way, and keep a nice linear Git history.
Rebasing is great whenever you're working on a feature branch, and the master branch has been updated.
-
This can happen when the two branches we're trying to merge have changes on the same line in the same file, or if one branch deleted a file that another branch modified, and so on.
Merge conflict - you have to decide from which branch to keep the change.
After:
git merge devGit will notify you about the merge conflict so you can manually remove the changes you don't want to keep, save them, and then:
git add updated_file.md git commit -m "Merge..." -
If we committed changes on the current branch that the branch we want to merge doesn't have, git will perform a no-fast-forward merge.
No-fast-forward merge - default behavior when current branch contains commits that the merging branch doesn't have.
Create a new commit which merges two branches together without modifying existing branches.
git merge dev -
fast-forward merge can happen when the current branch has no extra commits compared to the branch we’re merging.
Fast-forward merge - default behavior when the branch has all of the current branch's commits.
Doesn't create a new commit, thus doesn't modify existing branches.
git merge dev -
soft reset moves HEAD to the specified commit (or the index of the commit compared to HEAD), without getting rid of the changes that were introduced on the commits afterward!
Soft reset - points
HEADto the specified commit.Keeps changes that have been made since the new commit the
HEADpoints to, and keeps the modifications in the working directorygit reset --soft HEAD~2 git status -
git reset gets rid of all the current staged files and gives us control over where HEAD should point to.
Reset - way to get rid of unwanted commits. We have soft and hard reset
-
There are 6 actions we can perform on the commits we're rebasing
Interactive rebase - makes it possible to edit commits before rebasing.
Creates new commits for the edited commits which history has been changed.
6 actions (options) of interactive rebase:
reword: Change the commit messageedit: Amend this commitsquash: Meld commit into the previous commitfixup: Meld commit into the previous commit, without keeping the commit's log messageexec: Run a command on each commit we want to rebasedrop: Remove the commit
git rebase -i HEAD~3
-
-
www.linkedin.com www.linkedin.comLinkedIn2
-
Pharma, which is one of the biggest, richest, most rewarding and promising industries in the world. Especially now, when the pharmaceutical industry, including the FDA, allows R to be used the domain occupied in 110% by SAS.
Pharma industry is one of the most rewarding industries, especially now
-
CR is one of the most controlled industries in this world. It's insanely conservative in both used statistical methods and programming. Once a program is written and validated, it may be used for decades. There are SAS macros written in 1980 working still by today without any change. That's because of brilliant backward compatibility of the SAS macro-language. New features DO NOT cause the old mechanisms to be REMOVED. It's here FOREVER+1 day.
Clinical Research is highly conservative, making SAS macros applicable for decades. Unfortunately, that's not the same case with R
-
- Mar 2020
-
-
wearing simple face masks which exert a barrier function that blocks those big projectile droplets that land in the nose or throat may substantially reduce the production rate R, to an extent that may be comparable to social distancing and washing hands.
Most important message of the article
-
Make sure to discard or launder after use without touching the outward surface
What to do with a mask after usage
-
CDC suggests use of scarf by health care providers as last resort when no face masks are available
Use of scarf
-
avoiding large droplets, which cannot enter the lung anyway but land in the upper respiratory tracts, could be the most effective means to prevent infection. Therefore, surgical masks, perhaps even your ski-mask, bandanas or scarf
Wear a mask!
-
The molecular analysis also show that the SARS-Cov2 virus is active and replicates already in the nasopharynx, unlike other respiratory viruses that dwell in deeper regions of the lung.
-
Surprisingly, ACE2 expression in the lung is very low: it is limited to a few molecules per cell in the alveolar cells (AT2 cells) deep in the lung. But a just published paper by the Human Cell Atlas (HCA) consortium reports that ACE2 is highly expressed in some type of (secretory) cells of the inner nose!
Major route of viral entry is likely via large droplets that land in the nose — where expression of the viral entry receptor, ACE2 is highest. This is the transmission route that could be effectively blocked already by simple masks that provide a physical barrier.
-
SARS-Cov-2 virus, like any virus, must dock onto human cells using a key-lock principle, in which the virus presents the key and the cell the lock that is complementary to the key to enter the cell and replicate. For the SARS-Cov-2 virus, the viral surface protein “Spike protein S” is the “key” and it must fit snugly into the “lock” protein that is expressed (=molecularly presented) on the surface of the host cells. The cellular lock protein that the SARS-Cov-2 virus uses is the ACE2 protein
SARS-Cov-2 enters the host cell by docking with its Spike protein to the ACE2 (blue) protein in cell surfaces:
-
Filtering effect for small droplets (aerosols) by various masks; home-made of tea cloth, surgical mask (3M “Tie-on”) and a FFP2 (N95) respirator mask. The numbers are scaled to the reference of 100 (source of droplets) for illustrative purposes, calculated from the PF (protection factor) values
-
The tacit notion at the CDC that the alveolae are the destination site for droplets to deliver the virus load (the alveolae are after all the anatomical site of life-threatening pneumonia), has elevated the apparent importance of N95 masks and led to the dismissal of surgical masks.
Why N95 masks are much better over the surgical masks
-
In the case of the SARS-Cov-2 virus it is not known what the minimal infectious load is (number of viral particles needed to start the pathogenesis cascade that causes a clinical disease).
Minimal infectious load
-
Of course many aerosol droplets in the exhalation or cough spray may not contain the virus, but some will do.
-
droplets of a typical cough expulsion have a size distribution such that approximately half of the droplet are in the categories of aerosols, albeit they collectively represent only less than 1/100,000 of the expelled volume
Droplets of a typical cough
-
For airborne particles to be inspired and reach deep into the lung, through all the air ducts down to the alveolar cells where gas-exchange takes place, it has to be small
Only droplets < 10 um can reach to alveolae (deep into lung). Larger droplets stuck in the nose, throat, upper air ducts of the lung, trachea and large bronchia.
-
Droplets can (for this discussion) be crudely divided in two large categories based on size
2 categories of droplets:
a) Droplets < 10 um: upper size limit of
aerosol. Can float in the air/rooms by ventilation or winds and can be filtered (to 95%) by N95 favial masks (droplets < than 0.3 um). Here the surgical masks cannot help.b) Droplets > 10 um (reaching 100+ um): called as
spray droplets. Can be even visible by human from coughing/sneezing (0.1+ um). -
Droplet larger than aerosols, when exhaled (at velocity of <1m/s), evaporate or fall to the ground less than 1.5 m away. When expelled at high velocity through coughing or sneezing, especially larger droplets (> 0.1 micrometers), can be carried by the jet more than 2m or 6m, respectively, away.
-
The official recommendation by CDC, FDA and others that masks worn by the non-health-care professionals are ineffective is incorrect at three levels: In the logic, in the mechanics of transmission, and in the biology of viral entry.
-
Flattening the curve”. Effect of mitigating interventions that would decrease the initial reproduction rate R0 by 50% when implemented at day 25. Red curve is the course of numbers of infected individuals (”case”) without intervention. Green curve reflects the changed (”flattened”) curve after intervention. Day 0 (March 3, 2020) is the time at which 100 cases of infections were confirmed (d100 = 0).
If people would start wearing a mask:
-
-
-
Ancestor of all animals identified in Australian fossils
Summary:
- First ancestor of most animals, including humans, has been discovered—Ikaria wariootia had a mouth, anus, gut, and a bilaterian body plan.
- Bilateral symmetry was a critical step in evolution, enabling organisms to move purposefully, but so far the first organism to develop it wasn’t known.
- Ikaria wariootia was discovered through careful analysis of 555 million-year-old samples.
- It was a wormlike creature, up to 7mm (0.27in) long, with a distinct head and tail, as well as faintly grooved musculature.
- This discovery confirms what evolutionary biologists previously predicted.
-
-
www.instalki.pl www.instalki.pl
-
Połączona sieć komputerów działających w ramach inicjatywy Folding@Home przewyższyła swoją mocą obliczeniową najwydajniejsze siedem superkomputerów na świecie. Dobrze przeczytaliście: połączone w ramach F@H urządzenia dysponują mocą obliczeniową na poziomie 470 PetaFLOPS - wyższą od siedmiu najwydajniejszych superkomputerów na świecie razem wziętych! To trzy razy większa wydajność od tej, którą dysponuje najwydajniejszy obecnie superkomputer SUMMIT (149 PFLOPS).
Internauts build a supercomputer network stronger than 7 most efficient local supercomputers interconnected. The reason is to fight against COVID-19.
You can also join them by using Folding@home software
-
-
amitness.com amitness.com
-
This denotes the factorial of a number. It is the product of numbers starting from 1 to that number.
Exclamation in Python: $$x!$$ is written as:
x = 5 fact = 1 for i in range(x, 0, -1): fact = fact * i print(fact)it can be shortened as:
import math math.factorial(x)and the output is:
# 5*4*3*2*1 120 -
The hat gives the unit vector. This means dividing each component in a vector by it’s length(norm).
Hat in Python: $$\hat{x}$$ is written as:
x = [1, 2, 3] length = math.sqrt(sum([e**2 for e in x])) x_hat = [e/length for e in x]This makes the magnitude of the vector 1 and only keeps the direction:
math.sqrt(sum([e**2 for e in x_hat])) # 1.0 -
It gives the sum of the products of the corresponding entries of the two sequences of numbers.
Dot Product in Python: $$X.Y$$ is written as:
X = [1, 2, 3] Y = [4, 5, 6] dot = sum([i*j for i, j in zip(X, Y)]) # 1*4 + 2*5 + 3*6 # 32 -
It means multiplying the corresponding elements in two tensors. In Python, this would be equivalent to multiplying the corresponding elements in two lists.
Element wise multiplication in Python: $$z=x\odot y$$ is written as:
x = [[1, 2], [3, 4]] y = [[2, 2], [2, 2]] z = np.multiply(x, y)and results in:
[[2, 4]], [[6, 8]] -
This is basically exchanging the rows and columns.
Transpose in Python: $$X^T$$ is written as:
X = [[1, 2, 3], [4, 5, 6]] np.transpose(X)and results in:
[[1, 4], [2, 5], [3, 6]] -
This denotes a function which takes a domain X and maps it to range Y. In Python, it’s equivalent to taking a pool of values X, doing some operation on it to calculate pool of values Y.
Function in Python: $$f:X \rightarrow Y$$ is written as:
def f(X): Y = ... return YUsing
Rinstead ofXorYmeans we're dealing with real numbers: $$f:R \rightarrow R$$ then,R^2means we're dealing with d-dimensional vector of real numbers (in this case, example ofd=2isX = [1,2] -
This symbol checks if an element is part of a set.
Epilson in Python: $$3 \epsilon X$$ is written as:
X = {1,2,3} 3 in X -
The norm is used to calculate the magnitude of a vector. In Python, this means squaring each element of an array, summing them and then taking the square root.
Norm of vector in Python (it's like Pythagorean theorem): $$| x|$$ is written as:
x = [1, 2, 3] math.sqrt(x[0]**2 + x[1]**2 + x[2]**2) -
This symbol denotes the absolute value of a number i.e. without a sign.
Absolute value in Python: $$|x|$$ is written as:
x = 10 y = -20 abs(x) #10 abs(y) #20 -
In Python, it is equivalent to looping over a vector from index 0 to index N-1 and multiplying them.
PI in Python is the same as sigma, but you multiply (*) the numbers inside the for loop. $$\prod_{i=1}^Nx^i$$
-
we reuse the sigma notation and divide by the number of elements to get an average.
Average in Python: $$\frac{1}{N}\sum_{i=1}^Nx_i$$ is written as:
x = [1, 2, 3, 4, 5] result = 0 N = len(x) for i in range(n): result = result + x[i] average = result / N print(average)or it can be shortened:
x = [1, 2, 3, 4, 5] result = sum(x) / len(x) -
In Python, it is equivalent to looping over a vector from index 0 to index N-1
Sigma in Python: $$\sum_{i=1}^Nx_i$$ is written as:
x = [1, 2, 3, 4, 5] result = 0 N = len(x) for i in range(N): result = result + x[i] print(result)or it can be shortened:
x = [1, 2, 3, 4, 5] result = sum(x) -
2D vectors
2D vectors in Python: $$x_{ij}$$ are written as:
x = [ [10, 20, 30], [40, 50, 60] ] i = 0 j = 1 print(x[i][j]) # 20
Tags
Annotators
URL
-
-
en.wikipedia.org en.wikipedia.org
-
1–9–90 rule (sometimes 90–9–1 principle or the 89:10:1 ratio),[1] which states that in a collaborative website such as a wiki, 90% of the participants of a community only consume content, 9% of the participants change or update content, and 1% of the participants add content.
1% rule = 1% of the users create and 99% watch the content.
1-9-90 rule = 1% create, 9% modify and 90% watch
Tags
Annotators
URL
-
-
threadreaderapp.com threadreaderapp.com
-
Another nice SQL script paired with CRON jobs was the one that reminded people of carts that was left for more than 48 hours. Select from cart where state is not empty and last date is more than or equal to 48hrs.... Set this as a CRON that fires at 2AM everyday, period with less activity and traffic. People wake up to emails reminding them about their abandoned carts. Then sit watch magic happens. No AI/ML needed here. Just good 'ol SQL + Bash.
Another example of using SQL + CRON job + Bash to remind customers of cart that was left (again no ML needed here)
-
I will write a query like select from order table where last shop date is 3 or greater months. When we get this information, we will send a nice "we miss you, come back and here's X Naira voucher" email. The conversation rate for this one was always greater than 50%.
Sometimes SQL is much more than enough (you don't need ML)
-
-
datagenetics.com datagenetics.com
-
This volume of paper should be the same as the coaxial plug of paper on the roll.
Calculating volume of the paper roll: $$\mathbf{Lwt = \pi w(R^2 - r^2)} \~\ L = \text{length of the paper} \ w = \text{width of the paper} \ t = \text{thickness} \ R = \text{outer radius} \ r = \text{inner radius}$$ And that simplifies into a formula for R: $$\color{red} {\bf R = \sqrt{\frac{Lt}{\pi}+r^2}}$$
-
This shows the nonlinear relationship and how the consumption accelerates. The first 10% used makes just a 5% change in the diameter of the roll. The last 10% makes an 18.5% change.
Consumption of a toilet paper roll has a nonlinear relationship between the:
- y-axis (outer Radius of the roll (measured as a percentage of a full roll))
- x-axis (% of the roll consumed)
-
Toilet paper is typically supplied in rolls of perforated material wrapped around a central cardboard tube. There’s a little variance between manufacturers, but a typical roll is approximately 4.5” wide with an 5.25” external diameter, and a central tube of diameter 1.6” Toilet paper is big business (see what I did there?) Worldwide, approximately 83 million rolls are produced per day; that’s a staggering 30 billion rolls per year. In the USA, about 7 billion rolls a year are sold, so the average American citizen consumes two dozen rolls a year (two per month). Americans use 24 rolls per capita a year of toilet paper Again, it depends on the thickness and luxuriousness of the product, but the perforations typically divide the roll into approximately 1,000 sheets (for single-ply), or around 500 sheets (for double-ply). Each sheet is typically 4” long so the length of a (double-ply) toilet roll is approximately 2,000” or 167 feet (or less, if your cat gets to it).
Statistics on the type and use of toilet paper in the USA.
1" (inch) = 2.54 cm
Tags
Annotators
URL
-
-
-
In the interval scale, there is no true zero point or fixed beginning. They do not have a true zero even if one of the values carry the name “zero.” For example, in the temperature, there is no point where the temperature can be zero. Zero degrees F does not mean the complete absence of temperature. Since the interval scale has no true zero point, you cannot calculate Ratios. For example, there is no any sense the ratio of 90 to 30 degrees F to be the same as the ratio of 60 to 20 degrees. A temperature of 20 degrees is not twice as warm as one of 10 degrees.
Interval data:
- show not only order and direction, but also the exact differences between the values
- the distances between each value on the interval scale are meaningful and equal
- no true zero point
- no fixed beginning
- no possibility to calculate ratios (only add and substract)
- e.g.: temperature in Fahrenheit or Celsius (but not Kelvin) or IQ test
-
As the interval scales, Ratio scales show us the order and the exact value between the units. However, in contrast with interval scales, Ratio ones have an absolute zero that allows us to perform a huge range of descriptive statistics and inferential statistics. The ratio scales possess a clear definition of zero. Any types of values that can be measured from absolute zero can be measured with a ratio scale. The most popular examples of ratio variables are height and weight. In addition, one individual can be twice as tall as another individual.
Ratio data is like interval data, but with:
- absolute zero
- possibility to calculate ratio (e.g. someone can be twice as tall)
- possibility to not only add and subtract, but multiply and divide values
- e.g.: weight, height, Kelvin scale (50K is 2x hot as 25K)
Tags
Annotators
URL
-
-
paramaggarwal.substack.com paramaggarwal.substack.com
-
Javascript, APIs and Markup — this stack is all about finding middleground from the chaos of SSR+SPA. It is about stepping back and asking yourself, what parts of my page change and what parts don’t change?
JavaScript, APIs and Markup (JAM Stack) - middleground between SSR + SPA.
Advantages:
- The parts that don’t change often are pre-rendered on the server and saved to static HTML files. Anything else is implemented in JS and run on the client using API calls.
- Avoids too much data transfer (like the hydration data for SSR), therefore finds a good tradeoff to ship web content
- Allows to leverage the power and cost of Content delivery networks (CDNs) to effectively serve your content
- With serverless apps your APIs will never need a server to SSH into and manage
-
Somewhere on this path to render pages on the fly (SSR) and render pages on the client (SPA) we forgot about the performance of our webpages. We were trying to build apps. But the web is about presenting content first and foremost!
Website performance break with Client-side Rendering (SSR) and Single-page App (SPA)
-
We were not satisfied with the basic capabilities like bold and italics so we built CSS. Now, we wanted to modify some parts of the HTML/CSS in response to things like clicking things, so we implemented a scripting language to quickly specify such relations and have then run within the browser itself instead of a round trip to the server.
Birth of CSS - advanced styling
(history of websites)
-
And so was born PHP, it feels like a natural extension to HTML itself. You write your code between your HTML file itself and then be able to run those parts on the server, which further generate HTML and the final HTML gets send to the browser.This was extremely powerful. We could serve completely different pages to different users even though all of them access the same URL like Facebook. We could use a database on a server and store some data there, then based on some conditions use this data to modify the generated HTML and technically have an infinite number of pages available to serve (e-commerce).
Birth of PHP - way to serve different content under the same URL
-
-
-
TL;DR;
Don't use:
- No global vars.
- Declare variables using "var".
- Declare functions using "function" keyword.
- Avoid use "for" in loops.
- Array push, inmutability.
- Class.
- Use delete to remove a object atribute.
- Avoid nested if.
- else if.
- Heavy nesting.
- Avoid to add prototype to functions that could be used in a module.
Use:
- Common code in functions, follow D.R.Y principle.
- Shorcut notation.
- Spread operator over Object.assign (airbnb 3.8).
- Pascal case naming.
- Modularize your code in modules.
- const and let!.
- Literal syntax for object creation (airbnb 3.1).
- Computed property names when creating objects (airbnb 3.2).
- Property value shorthand (airbnb 3.4).
- Group your shorthand properties at the beginning of your objec (airbnb 3.5).
- use the literal syntax for array creation (airnbnb 4.1).
- Use array spreads ... to copy arrays. (airbnb 4.3).
- use spreads ... instead of, Array.from. (airbnb 4.4).
- Use return statements in array method callbacks (airbnb 4.7).
-
Don’t use iterators, prefer js higher-order functions instead of for / for..in
// bad const increasedByOne = []; for (let i = 0; i < numbers.length; i++) { increasedByOne.push(numbers[i] + 1); } // good const increasedByOne = []; numbers.forEach((num) => { increasedByOne.push(num + 1); }); -
Ternaries should not be nested and generally be single line expressions.
e.g.
const foo = maybe1 > maybe2 ? 'bar' : maybeNull;Also:
// bad const foo = a ? a : b; const bar = c ? true : false; const baz = c ? false : true; // good const foo = a || b; const bar = !!c; const baz = !c; -
Use === and !== over == and !=.
Bases on the Airbnb guide
-
Use JSDOC https://jsdoc.app/about-getting-started.html format.
Standardise your JavaScript comments:
- Use block comment
/** This is a description of the foo function. */ function foo() { } - Use JSDOC tag to describe a function: ```javascript /**
- Represents a book.
- @constructor
- @param {string} title - The title of the book.
- @param {string} author - The author of the book. */ function Book(title, author) { } ```
- Use block comment
-
When JavaScript encounters a line break without a semicolon, it uses a set of rules called Automatic Semicolon Insertion to determine whether or not it should regard that line break as the end of a statement, and (as the name implies) place a semicolon into your code before the line break if it thinks so. ASI contains a few eccentric behaviors, though, and your code will break if JavaScript misinterprets your line break.
Better place a semicolon (;) in the right place and do not rely on the Automatic Semicolon Insertion
e.g.
const luke = {}; const leia = {}; [luke, leia].forEach((jedi) => { jedi.father = 'vader'; });```
-
-
naukawpolsce.pap.pl naukawpolsce.pap.pl
-
Wykazali, iż u szczurów znajdujących się na diecie o niskiej zawartości tłuszczu wolniej dochodzi do niekorzystnych zmian strukturalnych i genetycznych w obrębie poszczególnych tkanek. Gryzonie, które jadły mniej, dłużej zachowywały młodość.
Low-calorie diet reduces inflammation, delays the onset of old age diseases, and generally prolongs life.
-
-
xd.adobe.com xd.adobe.com
-
If sentences contain eight words or less, readers understand 100 percent of the information.If sentences contain 43 words or longer, the reader’s comprehension drops to less than 10 percent.
<= 8 words <--- 100% understanding
.>= 43 words <--- <10% understanding
-
-
pythonspeed.com pythonspeed.com
-
from Docker Compose on a single machine, to Heroku and similar systems, to something like Snakemake for computational pipelines.
Other alternatives to Kubernetes:
- Docker Compose on a single machine
- Heroku and similar systems
- Snakemake for computational pipelines
-
if what you care about is downtime, your first thought shouldn’t be “how do I reduce deployment downtime from 1 second to 1ms”, it should be “how can I ensure database schema changes don’t prevent rollback if I screw something up.”
Caring about downtime
-
The features Kubernetes provides for reliability (health checks, rolling deploys), can be implemented much more simply, or already built-in in many cases. For example, nginx can do health checks on worker processes, and you can use docker-autoheal or something similar to automatically restart those processes.
Kubernetes' health checks can be replaced with nginx on worker processes + docker-autoheal to automatically restart those processes
-
Scaling for many web applications is typically bottlenecked by the database, not the web workers.
-
Kubernetes might be useful if you need to scale a lot. But let’s consider some alternatives
Kubernetes alternatives:
- cloud VMs with up to 416 vCPUs and 8 TiB RAM
- scale many web apps with Heroku
-
Distributed applications are really hard to write correctly. Really. The more moving parts, the more these problems come in to play. Distributed applications are hard to debug. You need whole new categories of instrumentation and logging to getting understanding that isn’t quite as good as what you’d get from the logs of a monolithic application.
Microservices stay as a hard nut to crack.
They are fine for an organisational scaling technique: when you have 500 developers working on one live website (so they can work independently). For example, each team of 5 developers can be given one microservice
-
you need to spin up a complete K8s system just to test anything, via a VM or nested Docker containers.
You need a complete K8s to run your code, or you can use Telepresence to code locally against a remote Kubernetes cluster
-
“Kubernetes is a large system with significant operational complexity. The assessment team found configuration and deployment of Kubernetes to be non-trivial, with certain components having confusing default settings, missing operational controls, and implicitly defined security controls.”
Deployment of Kubernetes is non-trivial
-
Before you can run a single application, you need the following highly-simplified architecture
Before running the simplest Kubernetes app, you need at least this architecture:
-
the Kubernetes codebase has significant room for improvement. The codebase is large and complex, with large sections of code containing minimal documentation and numerous dependencies, including systems external to Kubernetes.
As of March 2020, the Kubernetes code base has more than 580 000 lines of Go code
-
Kubernetes has plenty of moving parts—concepts, subsystems, processes, machines, code—and that means plenty of problems.
Kubernetes might be not the best solution in a smaller team
-
-
-
That makes sense, the new file gets created in the upper directory.
If you add a new file, such as with:
$ echonew file> merged/new_fileIt will be created in the
upperdirectory -
Combining the upper and lower directories is pretty easy: we can just do it with mount!
Combining lower and upper directories using mount:
$ sudo mount -t overlay overlay -o lowerdir=/home/bork/test/lower,upperdir=/home/bork/test/upper,workdir=/home/bork/test/work /home/bork/test/merged -
Overlay filesystems, also known as “union filesystems” or “union mounts” let you mount a filesystem using 2 directories: a “lower” directory, and an “upper” directory.
Docker doesn't make copies of images, but instead uses an overlay.
Overlay filesystems, let you mount a system using 2 directories:
- the lower directory (read-only)
- the upper directory (read and write).
When a process:
- reads a file, the overlayfs filesystem driver looks into the upper directory and if it's not present, it looks into the lower one
- writes a file, overlayfs will just use the upper directory
-
-
www.calnewport.com www.calnewport.com
-
Using Facebook ads, the researchers recruited 2,743 users who were willing to leave Facebook for one month in exchange for a cash reward. They then randomly divided these users into a Treatment group, that followed through with the deactivation, and a Control group, that was asked to keep using the platform.
The effects of not using Facebook for a month:
- on average another 60 free mins per day
- small but significant improvement in well-being, and in particular in self-reported happiness, life satisfaction, depression and anxiety
- participants were less willing to use Facebook from now
- the group was less likely to follow politics
- deactivation significantly reduced polarization of views on policy issues and a measure of exposure to polarizing news
- 80% agreed that the deactivation was good for them
-
-
news.uark.edu news.uark.edu
-
In addition to the results on creativity, caffeine did not significantly affect working memory, but test subjects who took it did report feeling less sad.
Coffee also:
.+ happier
-
While the cognitive benefits of caffeine — increased alertness, improved vigilance, enhanced focus and improved motor performance — are well established, she said, the stimulant’s affect on creativity is less known.
Coffee: .+ alertness
.+ vigilance
.+ focus
.+ motor performance
? creativity
-