if you imported moduleX (note: imported, not directly executed), its name would be package.subpackage1.moduleX. If you imported moduleA, its name would be package.moduleA. However, if you directly run moduleX from the command line, its name will instead be __main__, and if you directly run moduleA from the command line, its name will be __main__. When a module is run as the top-level script, it loses its normal name and its name is instead __main__.

A file is loaded as the top-level script if you execute it directly, for instance by typing python myfile.py on the command line. It is loaded as a module if you do python -m myfile, or if it is loaded when an import statement is encountered inside some other file.

After working on the problem for a while, we boiled it down to a 4-turn (2 per player), 9 roll (including doubles) game. Detail on each move given below. If executed quickly enough, this theoretical game can be played in 21 seconds (see video below).

The majority of Python packaging tools also act as virtualenv managers to gain the ability to isolate project environments. But things get tricky when it comes to nested venvs: One installs the virtualenv manager using a venv encapsulated Python, and create more venvs using the tool which is based on an encapsulated Python. One day a minor release of Python is released and one has to check all those venvs and upgrade them if required. PEP 582, on the other hand, introduces a way to decouple the Python interpreter from project environments. It is a relative new proposal and there are not many tools supporting it (one that does is pyflow), but it is written with Rust and thus can't get much help from the big Python community. For the same reason it can't act as a PEP 517 backend.

After suggestions from comments below, I read A Philosophy of Software Design (2018) by John Ousterhout and found it to be a much more positive experience. I would be happy to recommend it over Clean Code.

If you insist on using Git and insist on tracking many large files in version control, you should definitely consider LFS. (Although, if you are a heavy user of large files in version control, I would consider Plastic SCM instead, as they seem to have the most mature solution for large files handling.)

In Mercurial, use of LFS is a dynamic feature that server/repo operators can choose to enable or disable whenever they want. When the Mercurial server sends file content to a client, presence of external/LFS storage is a flag set on that file revision. Essentially, the flag says the data you are receiving is an LFS record, not the file content itself and the client knows how to resolve that record into content.

If you adopt LFS today, you are committing to a) running an LFS server forever b) incurring a history rewrite in the future in order to remove LFS from your repo, or c) ceasing to provide an LFS server and locking out people from using older Git commits.

:/<text>, e.g. :/fix nasty bug A colon, followed by a slash, followed by a text, names a commit whose commit message matches the specified regular expression.

git worktree add <path> automatically creates a new branch whose name is the final component of <path>, which is convenient if you plan to work on a new topic. For instance, git worktree add ../hotfix creates new branch hotfix and checks it out at path ../hotfix.

LocationManager can provide a GPS location (lat,long) every second. Meanwhile, TelephonyManager gives the cellID=(mmc,mcc,lac,cid) the radio is currently camping on. A cellID database[1], allows to know the (lat,long) of each CellID. What is left is to draw the itinerary (in red) and, for each second, a cellID-color-coded connection to the cell.

For instance, when someone is not confident in what they are saying or know that they are lying, they may take a step backwards as if stepping back from the situation.

People with ADHD are highly sensitive to micro expressions.

There are seven expressions that are universal to all humans no matter which country they are born: anger, fear, disgust, sadness, happiness, surprise, and contempt.

A macro expression is the normal facial expression you notice that lasts between ½ a second to 4 seconds. These are fairly easy to notice and tend to match the tone and content of what is being said.A micro expression is an involuntary facial expressions that lasts less than half a second. These expressions are often missed altogether but they reveal someones true feelings about what they are saying.

From an employers perspective, I believe there are many advantages:

The research found that working 55 hours or more a week was associated with a 35% higher risk of stroke and a 17% higher risk of dying from heart disease, compared with a working week of 35 to 40 hours.

What I did have going for me was what might be considered hacker sensibilities - lack of fear of computers in general, and in trying unknown things. A love of exploring, learning by experimenting, putting stuff together on the fly.

The only problem is that Kubeflow Pipelines must be deployed on a Kubernetes Cluster. You will struggle with permissions, VPC and lots of problems to deploy and use it if you are in a small company that uses sensitive data, which makes it a bit difficult to be adoptedVertex AI solves this problem with a managed pipeline runner: you can define a Pipeline and it will executed it, being responsible to provision all resources, store all the artifacts you want and pass them through each of the wanted steps.

Kubeflow Pipelines comes to solve this problem. KFP, for short, is a toolkit dedicated to run ML Workflows (as experiments for model training) on Kubernetes, and it does it in a very clever way:Along with other ways, Kubeflow lets us define a workflow as a series of Python functions that pass results, and Artifacts for one another.For each Python function, we can define dependencies (for libs used) and Kubeflow will create a container to run each function in an isolated way, and passing any wanted object to a next step on the workflow. We can set needed resources, (as memory or GPUs) and it will provision them for our workflow step. It feels like magic.Once you’ve ran your pipeline, you will be able to see it in a nice UI, like this:

Vertex AI came from the skies to solve our MLOps problem with a managed — and reasonably priced—alternative. Vertex AI comes with all the AI Platform classic resources plus a ML metadata store, a fully managed feature store, and a fully managed Kubeflow Pipelines runner.

Method 1: We can grab the PDF Versions of Google’s TotT episodes or create our own posters that are more relevant to the company and put them in places where both developers and testers can’t be ignored.Method 2 : We can initiate something called ‘Tip of the day’ Mailing System from Quality Engineering Department.

They started to write flyers about everything from dependency injection to code coverage and then regularly plaster the bathrooms in all over Google with each episode, almost 500 stalls worldwide.

Dogfooding → Internal adoption of software that is not yet released. The phrase “eating your own dogfood” is meant to convey the idea that if you make a product to sell to someone else, you should be willing to use it yourself to find out if it is any good.

It's Faster

It's even more discoverable if your shell has smart tab completion: for example, on my current project, if you enter the aws/ directory and type make<TAB>, you'll see a list that includes things like docker-login, deploy-dev and destroy-sandbox.

Variables. Multiple lines. No more escaped quotation marks. Comments.

People might think I’m not experiencing new things, but I think the secret to a good life is to enjoy your work. I could never stay indoors and watch TV. I hear London is a place best avoided. I think living in a city would be terrible – people living on top of one another in great tower blocks. I could never do it. Walking around the farm fills me with wonder. What makes my life is working outside, only going in if the weather is very bad.

pronunciation cues in an individual’s speech communicate their social status more accurately than the content of their speech.

reciting seven random words is sufficient to allow people to discern the speaker’s social class with above-chance accuracy.

Cookiecutter takes a source directory tree and copies it into your new project. It replaces all the names that it finds surrounded by templating tags {{ and }} with names that it finds in the file cookiecutter.json. That’s basically it. [1]

In short, MLflow makes it far easier to promote models to API endpoints on various cloud vendors compared to Kubeflow, which can do this but only with more development effort.

Bon Appétit?

MLflow is a single python package that covers some key steps in model management. Kubeflow is a combination of open-source libraries that depends on a Kubernetes cluster to provide a computing environment for ML model development and production tools.

I decided to embrace imperfection and use it as the unifying theme of the album.

I start by sampling loops from songs I like, and then use the concatenative synthesis and source separations tools to extract interesting musical ideas (e.g., melodies, percussion, ambiance). These results can be used directly, with a little FX (Replica XT, Serum FX), or translated into MIDI and resynthesized (Serum, VPS Avenger). This gives me the building blocks of the song (the tracks), each a few bars long, which I carefully combine using Ableton Live's session view into something that sounds "good". Later, each track is expanded by sampling more loops from the original songs until I have plenty of content to create something with enough structure to be called a "song".

Another extremely interesting and underrated task in the audio domain is translating audio into MIDI notes. Melodyne is state of the art in the area, but Ableton's Convert Audio to Midi works really well too.

imperfect separations are sometimes the most interesting, especially the results from models trained to separate something that isn't in the source sound (e.g., extract vocals from an instrumental track)

Sound Source Separation

concatenative sound synthesis. The main idea is simple: given a database of sound snippets, concatenate or mix these sounds in order to create new and original ones.

Calibri has been the default font for all things Microsoft since 2007, when it stepped in to replace Times New Roman across Microsoft Office. It has served us all well, but we believe it’s time to evolve. To help us set a new direction, we’ve commissioned five original, custom fonts to eventually replace Calibri as the default.

To summarize, implementing ML in a production environment doesn't only mean deploying your model as an API for prediction. Rather, it means deploying an ML pipeline that can automate the retraining and deployment of new models. Setting up a CI/CD system enables you to automatically test and deploy new pipeline implementations. This system lets you cope with rapid changes in your data and business environment. You don't have to immediately move all of your processes from one level to another. You can gradually implement these practices to help improve the automation of your ML system development and production.

🐛 (home, components): Resolve issue with modal collapses 🚚 (home): Move icons folder ✨ (newsletter): Add Newsletter component

feat(home, components): Add login modal fix(home, components): Resolve issue with modal collapses chore(home): Move icons folder

When audio quality is high (vs low), people judge the content as better and more important. They also judge the speaker as more intelligent, competent, and likable.In an experiment, people rated a physicist’s talk at a scientific conference as 19.3% better when they listened to it in high quality audio vs slightly distorted, echo-prone audio.

Just write your markdown as text/plain, email clients are broken.

Bring up the board that shows the status of every item the team is working on.Starting in the right-most column, get an update for each ticket in the column.Be sure to ask: “What’s needed to move this ticket to the next stage of progress?”Highlight blockers that people bring up and define what’s needed to unblock work.Move to the next column to the left and get updates for each ticket in that column.Continue until you get to the left-most column.

More people brings more status updates.More updates mean more information that others won’t care about.The team may be working on multiple projects at once.If the team has customers, ad-hoc work will regularly come up.

I start thinking about my update to prove I should keep my jobPeople zone out when a teammate starts talking about how they worked on something that doesn’t affect me.It’s finally my turn to give an update. No one is listening except for the facilitator.We go over our time and end when the next team starts lurking outside the room.

What did I complete yesterday that contributed to the team?What do I plan to complete today to contribute to the team?What impediments do I see that prevent me or the team from achieving its goals?

On the median case, Colab is going to assign users a K80, and the GTX 1080 is around double the speed, which does not stack up particularly well for Colab. However, on occasion, when a P100 is assigned, the P100 is an absolute killer GPU (again, for FREE).

With Spark 3.1, the Spark-on-Kubernetes project is now considered Generally Available and Production-Ready.

The key libraries of TFX are as follows

The console is a killer SQLite feature for data analysis: more powerful than Excel and more simple than pandas. One can import CSV data with a single command, the table is created automatically

This is not a problem if your DBMS supports SQL recursion: lots of data can be generated with a single query. The WITH RECURSIVE clause comes to the rescue.

Interrupt if: T3F > RTW (IO + T3E)

When you seek advice, first write down everything you’ve tried.

If you’re stuck for over an hour, seek help.

Simple … a single Linode VPS.

When peeking at your brain may help with mental illness

yyyy-mm-dd hh:mm:ss

We use Prometheus to collect time-series metrics and Grafana for graphs, dashboards, and alerts.

large machine learning job spans many nodes and runs most efficiently when it has access to all of the hardware resources on each node. This allows GPUs to cross-communicate directly using NVLink, or GPUs to directly communicate with the NIC using GPUDirect. So for many of our workloads, a single pod occupies the entire node.

We use Kubernetes mainly as a batch scheduling system and rely on our autoscaler to dynamically scale up and down our cluster — this lets us significantly reduce costs for idle nodes, while still providing low latency while iterating rapidly.

For high availability, we always have at least 2 masters, and set the --apiserver-count flag to the number of apiservers we’re running (otherwise Prometheus monitoring can get confused between instances).

We’ve increased the max etcd size with the --quota-backend-bytes flag, and the autoscaler now has a sanity check not to take action if it would terminate more than 50% of the cluster.

Another helpful tweak was storing Kubernetes Events in a separate etcd cluster, so that spikes in Event creation wouldn’t affect performance of the main etcd instances.

The root cause: the default setting for Fluentd’s and Datadog’s monitoring processes was to query the apiservers from every node in the cluster (for example, this issue which is now fixed). We simply changed these processes to be less aggressive with their polling, and load on the apiservers became stable again:

We then moved the etcd directory for each node to the local temp disk, which is an SSD connected directly to the instance rather than a network-attached one. Switching to the local disk brought write latency to 200us, and etcd became healthy!

When should you end a conversation? Probably sooner than you think

Why we’re so bad at daydreaming, and how to fix it

The newest generation of image codecs—in particular AVIF and JPEG XL—are a major improvement of the old JPEG and PNG codecs. To be sure, JPEG 2000 and WebP also compress more effectively and offer more features, yet the overall gain is not significant and consistent enough to warrant fast and widespread adoption. AVIF and JPEG XL will do much better—at least that’s what I hope.

Scientists break through the wall of sleep to the untapped world of dreams

'Night owls' may be twice as likely as morning 'larks' to underperform at work

Consider the amount of data and the speed of the data, if low latency is your priority use Akka Streams, if you have huge amounts of data use Spark, Flink or GCP DataFlow.

As we mentioned before, the majority of machine learning implementations are based on running model serving as a REST service, which might not be appropriate for the high volume data processing or usage of the streaming system, which requires re coding/starting systems for model update, for example, TensorFlow or Flink. Model as Data is a great fit for big data pipelines. For online inference, it is quite easy to implement, you can store the model anywhere (S3, HDFS…), read it into memory and call it.

The most common way to deploy a trained model is to save into the binary format of the tool of your choice, wrap it in a microservice (for example a Python Flask application) and use it for inference.

When we are providing our API endpoint to frontend team we need to ensure that we don’t overwhelm them with preprocessing technicalities.We might not always have a Python backend server (eg. Node.js server) so using numpy and keras libraries, for preprocessing, might be a pain.If we are planning to serve multiple models then we will have to create multiple TensorFlow Serving servers and will have to add new URLs to our frontend code. But our Flask server would keep the domain URL same and we only need to add a new route (a function).Providing subscription-based access, exception handling and other tasks can be carried out in the Flask app.

Next, imagine you have more models to deploy. You have three optionsLoad the models into the existing cluster — having one cluster serve all models.Spin up a new cluster to serve each model — having multiple clusters, one cluster serves one model.Combination of 1 and 2 — having multiple clusters, one cluster serves a few models.The first option would not scale, because it’s just not possible to load all models into one cluster as the cluster has limited resources.The second option will definitely work but it doesn’t sound like an effective process, as you need to create a set of resources every time you have a new model to deploy. Additionally, how do you optimize the usage of resources, e.g., there might be unutilized resources in your clusters that could potentially be shared by the rest.The third option looks promising, you can manually choose the cluster to deploy each of your new models into so that all the clusters’ resource utilization is optimal. The problem is you have to manuallymanage it. Managing 100 models using 25 clusters can be a challenging task. Furthermore, running multiple models in a cluster can also cause a problem as different models usually have different resource utilization patterns and can interfere with each other. For example, one model might use up all the CPU and the other model won’t be able to serve anymore.Wouldn’t it be better if we had a system that automatically orchestrates model deployments based on resource utilization patterns and prevents them from interfering with each other? Fortunately, that is exactly what Kubernetes is meant to do!

If you’re running lots of deployments of models then it becomes important to record which versions were deployed and when. This is needed to be able to go back to specific versions. Model registries help with this problem by providing ways to store and version models.

Here is a quick recap table of every technology we discussed in this blog post.

Can the Brain Resist the Group Opinion?

The benefits of applying GitOps best practices are far reaching and provide:

GitOps is a way to do Kubernetes cluster management and application delivery.  It works by using Git as a single source of truth for declarative infrastructure and applications. With GitOps, the use of software agents can alert on any divergence between Git with what's running in a cluster, and if there's a difference, Kubernetes reconcilers automatically update or rollback the cluster depending on the case. With Git at the center of your delivery pipelines, developers use familiar tools to make pull requests to accelerate and simplify both application deployments and operations tasks to Kubernetes.

It was actually very common to have one Dockerfile to use for development (which contained everything needed to build your application), and a slimmed-down one to use for production, which only contained your application and exactly what was needed to run it. This has been referred to as the “builder pattern”.

volumes are often a better choice than persisting data in a container’s writable layer, because a volume does not increase the size of the containers using it, and the volume’s contents exist outside the lifecycle of a given container.

after 4 decades, we can observe that not much has changed beside learning how to measure the “figuring out” time.

1979 in a book by Zelkowitz, Shaw, and Gannon entitled Principles of software engineering and design. It said that most of the development time was spent on maintenance (67%).

I can't recommend the Data Engineer career enough for junior developers. It's how I started and what I pursued for 6 years (and I would love doing it again), and I feel like it gave me such an incredible foundation for future roles :- Actually big data (so, not something you could grep...) will trigger your code in every possible way. You quickly learn that with trillions of input, the probabily to reach a bug is either 0% or 100%. In turn, you quickly learn to write good tests.- You will learn distributed processing at a macro level, which in turn enlighten your thinking at a micro level. For example, even though the order of magnitudes are different, hitting data over network versus on disk is very much like hitting data on disk versus in cache. Except that when the difference ends up being in hours or days, you become much more sensible to that, so it's good training for your thoughts.- Data engineering is full of product decisions. What's often called data "cleaning" is in fact one of the import product decisions made in a company, and a data engineer will be consistently exposed to his company product, which I think makes for great personal development- Data engineering is fascinating. In adtech for example, logs of where ads are displayed are an unfiltered window on the rest of humanity, for the better or the worse. But it definitely expands your views on what the "average" person actually does on its computer (spoiler : it's mainly watching porn...), and challenges quite a bit what you might think is "normal"- You'll be plumbing technologies from all over the web, which might or might not be good news for you.So yeah, data engineering is great ! It's not harder than other specialties for developers, but imo, it's one of the fun ones !

We recommend the Alpine image as it is tightly controlled and small in size (currently under 5 MB), while still being a full Linux distribution. This is fine advice for Go, but bad advice for Python, leading to slower builds, larger images, and obscure bugs.

If a service can run without privileges, use USER to change to a non-root user. This is excellent advice. Running as root exposes you to much larger security risks, e.g. a CVE in February 2019 that allowed escalating to root on the host was preventable by running as a non-root user. Insecure: However, the official documentation also says: … you should use the common, traditional port for your application. For example, an image containing the Apache web server would use EXPOSE 80. In order to listen on port 80 you need to run as root. You don’t want to run as root, and given pretty much every Docker runtime can map ports, binding to ports <1024 is completely unnecessary: you can always map port 80 in your external port. So don’t bind to ports <1024, and do run as a non-privileged user.

Multi-stage builds allow you to drastically reduce the size of your final image, without struggling to reduce the number of intermediate layers and files. This is true, and for all but the simplest of builds you will benefit from using them. Bad: However, in the most likely image build scenario, naively using multi-stage builds also breaks caching. That means you’ll have much slower builds.

layer caching is great: it allows for faster builds and in some cases for smaller images. Insecure: However, it also means you won’t get security updates for system packages. So you also need to regularly rebuild the image from scratch.

Different data sources are better suited for different types of data transformations and provide access to different data quantities at different freshnesses

MLOps platforms like Sagemaker and Kubeflow are heading in the right direction of helping companies productionize ML. They require a fairly significant upfront investment to set up, but once properly integrated, can empower data scientists to train, manage, and deploy ML models. 

…Well, deploying ML is still slow and painful

Music gives the brain a crucial connective advantage

Downloading a pretrained model off the Tensorflow website on the Iris dataset probably is no longer enough to get that data science job. It’s clear, however, with the large number of ML engineer openings that companies often want a hybrid data practitioner: someone that can build and deploy models. Or said more succinctly, someone that can use Tensorflow but can also build it from source.

When machine learning become hot 🔥 5-8 years ago, companies decided they need people that can make classifiers on data. But then frameworks like Tensorflow and PyTorch became really good, democratizing the ability to get started with deep learning and machine learning. This commoditized the data modelling skillset. Today, the bottleneck in helping companies get machine learning and modelling insights to production center on data problems.

Overall the consolidation made the differences even more pronounced! There are ~70% more open data engineer than data scientist positions. In addition, there are ~40% more open ML engineer than data scientist positions. There are also only ~30% as many ML scientist as data scientist positions.

Data scientist: Use various techniques in statistics and machine learning to process and analyse data. Often responsible for building models to probe what can be learned from some data source, though often at a prototype rather than production level. Data engineer: Develops a robust and scalable set of data processing tools/platforms. Must be comfortable with SQL/NoSQL database wrangling and building/maintaining ETL pipelines. Machine Learning (ML) Engineer: Often responsible for both training models and productionizing them. Requires familiarity with some high-level ML framework and also must be comfortable building scalable training, inference, and deployment pipelines for models. Machine Learning (ML) Scientist: Works on cutting-edge research. Typically responsible for exploring new ideas that can be published at academic conferences. Often only needs to prototype new state-of-the-art models before handing off to ML engineers for productionization.

I scraped the homepage URLs of every YC company since 2012, producing an initial pool of ~1400 companies. Why stop at 2012? Well, 2012 was the year that AlexNet won the ImageNet competition, effectively kickstarting the machine learning and data-modelling wave we are now living through. It’s fair to say that this birthed some of the earliest generations of data-first companies. From this initial pool, I performed keyword filtering to reduce the number of relevant companies I would have to look through. In particular, I only considered companies whose websites included at least one of the following terms: AI, CV, NLP, natural language processing, computer vision, artificial intelligence, machine, ML, data. I also disregarded companies whose website links were broken.

There are 70% more open roles at companies in data engineering as compared to data science. As we train the next generation of data and machine learning practitioners, let’s place more emphasis on engineering skills.

When you think about it, a data scientist can be responsible for any subset of the following: machine learning modelling, visualization, data cleaning and processing (i.e. SQL wrangling), engineering, and production deployment.

A big reason for GPUs popularity in Machine Learning is that they are essentially fast matrix multiplication machines. Deep Learning is essentially matrix multiplication.

Unity ML agents is a way for you to turn a video game into a Reinforcement Learning environment.

Haskell is the best functional programming language in the world and Neural Networks are functionsThis is the main motivation behind Hasktorch which lets you discover new kinds of Neural Network architectures by combining functional operators

I can’t think of a single large company where the NLP team hasn’t experimented with HuggingFace. They add new Transformer models within days of the papers being published, they maintain tokenizers, datasets, data loaders, NLP apps. HuggingFace has created multiple layers of platforms that each could be a compelling company in its own right.

Keras is a user centric library whereas Tensorflow especially Tensorflow 1.0 is a machine centric library. ML researchers think in terms of terms of layers, automatic differentiation engines think in terms of computational graphs.As far as I’m concerned my time is more valuable than the cycles of a machine so I’d rather use something like Keras.

A matrix is a linear map but linear maps are far more intuitive to think about than matrices

Graduate Student Descent

BERT engineer is now a full time job. Qualifications include:Some bash scriptingDeep knowledge of pip (starting a new environment is the suckier version of practicing scales)Waiting for new HuggingFace models to be releasedWatching Yannic Kilcher’s new Transformer paper the day it comes outRepeating what Yannic said at your team reading group

“Useful” Machine Learning research on all datasets has essentially reduced to making Transformers faster, smaller and scale to longer sequence lengths.

The best people in empirical fields are typically those who have accumulated the biggest set of experiences and there’s essentially two ways to do this.Spend lots of time doing itGet really good at running many concurrent experiments

I often get asked by young students new to Machine Learning, what math do I need to know for Deep Learning and my answer is Matrix Multiplication and Derivatives of square functions.

Slack is the best digital watercooler in the world but it’s a terrible place to collaborate - long winded disagreements should happen over Zoom and collaborations should happen in a document instead.

Even if you really love meetings, you can only attend about 10h of them per day but a single well written document will continue being read even while you’re sleeping.It’s unlikely that Christianity would have garnered millions of followers if Jesus Christ had to get on a “quick call” with each new potential prospect.Oral cultures don’t scale.

Friendship doesn’t scale easily to large groups and especially not forcibly. Friendships happen spontaneously at smaller scales.

Remote work forces a writing culture. In person work encourages an oral culture

He attributes his longevity to being an avid reader, his spirituality, diet, frequent exercise, steadfast persistence and discipline.

- To biznes, eksperci i obywatele są prawdziwymi twórcami polskiego ekosystemu AI. Państwo powinno przede wszystkim ich wspierać. W najbliższym czasie planujemy serię otwartych spotkań z każdą z tych grup, na których będziemy wspólnie pracować nad uszczegółowieniem – zapowiedział Antoni Rytel, wicedyrektor GovTech Polska. - Oprócz tego, specjalne zespoły będą zapewniać ciągłe wsparcie wszystkim tym podmiotom. Uruchomimy też kanał bieżącego zgłaszania pomysłów technicznych i organizacyjnych wspierających rozwój AI w naszym kraju – dodał.

W okresie krótkoterminowym decydujące dla sukcesu polityki sztucznej inteligencji będzie ochrona talentów posiadających zdolności modelowania wiedzy i analityki danych w systemach AI oraz wsparcie dla rozwoju własności intelektualnej wytwarzanej w naszym kraju – dodaje Robert Kroplewski, pełnomocnik ministra cyfryzacji ds. społeczeństwa informacyjnego.

Dokument określa działania i cele dla Polski w perspektywie krótkoterminowej (do 2023 r.), średnioterminowej (do 2027 r.) i długoterminowej (po 2027 r.). Podzieliliśmy je na sześć obszarów: AI i społeczeństwo – działania, które mają uczynić z Polski jednego z większych beneficjentów gospodarki opartej na danych, a z Polaków - społeczeństwo świadome konieczności ciągłego podnoszenia kompetencji cyfrowych. AI i innowacyjne firmy – wsparcie polskich przedsiębiorstw AI, m.in. tworzenie mechanizmów finansowania ich rozwoju, współpracy start up-ów z rządem. AI i nauka – wsparcie polskiego środowiska naukowego i badawczego w projektowaniu interdyscyplinarnych wyzwań lub rozwiązań w obszarze AI, m.in. działania mające na celu przygotowanie kadry ekspertów AI. AI i edukacja – działania podejmowane od kształcenia podstawowego, aż do poziomu uczelni wyższych – programy kursów dla osób zagrożonych utratą pracy na skutek rozwoju nowych technologii, granty edukacyjne. AI i współpraca międzynarodowa – działania na rzecz wsparcia polskiego biznesu w zakresie AI oraz rozwój technologii na arenie międzynarodowej. AI i sektor publiczny – wsparcie sektora publicznego w realizacji zamówień na rzecz AI, lepszej koordynacji działań oraz dalszym rozwoju takich programów jak GovTech Polska.

Rozwój AI w Polsce zwiększy dynamikę PKB o nawet 2,65 pp w każdym roku. Do 2030 r. pozwoli zautomatyzować ok. 49% czasu pracy w Polsce, generując jednocześnie lepiej płatne miejsca pracy w kluczowych sektorach.

Elon Musk declared that we are marching towards a near future without language, in which we’ll be able to communicate directly mind-to-mind through neural links. ‘Our brain spends a lot of effort compressing a complex concept into words,’ he said in a recent interview, ‘and there’s a lot of loss of information that occurs when compressing a complex concept into words.’

Mute inner speech can appear as an inner dialogue as well, but its truncated form encourages us to create a ‘secret’ abbreviated language and deploy mental shortcuts. By forcing us to articulate ourselves more fully, self-talk summons up the image of an imagined listener or interrogator more vividly. In this way, it allows us to question ourselves more critically by adopting an external perspective on our ideas, and so to consider shortcomings in our arguments – all while using our own speech.

Contemporary theories in cognition and the science of learning reaffirm Kleist’s speculations, and show how self-talk contributes not only to motivation and emotional regulation, but also to some higher cognitive functions such as developing metacognition and reasoning.

It’s no coincidence that we walk when we need to think: evidence shows that movement enhances thinking and learning, and both are activated in the same centre of motor control in the brain. In the influential subfield of cognitive science concerned with ‘embodied’ cognition, one prominent claim is that actions themselves are constitutive of cognitive processes. That is, activities such as playing a musical instrument, writing, speaking or dancing don’t start in the brain and then emanate out to the body as actions; rather, they entail the mind and body working in concert as a creative, integrated whole, unfolding and influencing each other in turn. It’s therefore a significant problem that many of us are trapped in work and study environments that don’t allow us to activate these intuitive cognitive muscles, and indeed often even encourage us to avoid them.

Did you know that everything you can do in VBA can also be done in Python? The Excel Object Model is what you use when writing VBA, but the same API is available in Python as well.See Python as a VBA Replacement from the PyXLL documentation for details of how this is possible.

You can write Excel worksheet functions in your Jupyter notebook too. This is a really great way of trying out ideas without leaving Excel to go to a Python IDE.

Use the magic function “%xl_get” to get the current Excel selection in Python. Have a table of data in Excel? Select the top left corner (or the whole range) and type “%xl_get” in your Jupyter notebook and voila! the Excel table is now a pandas DataFrame.

to run Python code in Excel you need the PyXLL add-in. The PyXLL add-in is what lets us integrate Python into Excel and use Python instead of VBA

New Quantum Algorithms Finally Crack Nonlinear Equations

2020 is a strong year for NLP. Companies like Hugging Face 🤗 , spaCy, Rasa became stronger and also more educational which ultimately drove a huge NLP revolution (at even Industry-level which is quite hard usually).

Teen dislike of physical appearance strong predictor of depression in early adulthood

To the brain, reading computer code is not the same as reading language Neuroscientists find that interpreting code activates a general-purpose brain network, but not language-processing centers.

blue light has been found to have the strongest impact on your natural production of melatonin, filtering it out when you’re using devices at night can make it easier to fall asleep, according to a recent study published in the Journal of Applied Psychology. But even here, experts see a more mundane connection between devices and comfort: “Most of the negative impact of electronics on sleep and on your circadian clock is not due to the light. It’s due to the fact that these things are engineered to keep you awake and entertained and engaged in what you’re doing for an extended period of time,”

Reynolds encourages following the 20-20-20 rule: “Take a 20-second break to view something 20 feet away every 20 minutes.”

The American Academy of Ophthalmology asserts that dry eyes are more likely to be the result of decreased blinking. The AAO asserts that people blink 15 times per minute in normal circumstances (“normal” being relative, here) but only five to seven times per minute while staring at a screen. Solutions for preventing dry eyes can be as simple as reminding yourself to blink, or using artificial tear drops throughout your day as a preventive measure. It’s also possible that you’re blinking even less if your superclose to your screen.

Once one has learned how to concentrate, the next thing to learn is to be able to concentrate on the right thing.

Review your own code first Write a clear changelist description Automate the easy stuff Answer questions with the code itself Narrowly scope changes Separate functional and non-functional changes Break up large changelists Respond graciously to critiques Be patient when your reviewer is wrong Communicate your responses explicitly Artfully solicit missing information Award all ties to your reviewer Minimize lag between rounds of review

TFX and Tensorflow run anywhere Python runs, and that’s a lot of places

since TFX and Tensorflow were built by Google, it has first-class support in the Google Cloud Platform.

After consideration, you decide to use Python as your programming language, Tensorflow for model building because you will be working with a large dataset that includes images, and Tensorflow Extended (TFX), an open-source tool released and used internally at Google, for building your pipelines.

These components has built-in support for ML modeling, training, serving, and even managing deployments to different targets.

Most data scientists feel that model deployment is a software engineering task and should be handled by software engineers because the required skills are more closely aligned with their day-to-day work. While this is somewhat true, data scientists who learn these skills will have an advantage, especially in lean organizations. Tools like TFX, Mlflow, Kubeflow can simplify the whole process of model deployment, and data scientists can (and should) quickly learn and use them. 

TFX Component called TensorFlow Model Analysis (TFMA) allows you to easily evaluate new models against current ones before deployment. 

In general, for smaller businesses like startups, it is usually cheaper and better to use managed cloud services for your projects. 

This has its pros and cons and may depend on your use case as well. Some of the pros to consider when considering using managed cloud services are:

The data is already in the cloud, so it may be better to build your ML system in the cloud. You’ll get better latency for I/O, easy scaling as data becomes larger (hundreds of gigabytes), and quick setup and configuration for any additional GPUs and TPUs. 

ML projects are never static. This is part of engineering and design that must be considered from the start. Here you should answer questions like:

The choice of framework is very important, as it can decide the continuity, maintenance, and use of a model. In this step, you must answer the following questions:

These questions are important as they will guide you on what frameworks or tools to use, how to approach your problem, and how to design your ML model.

you should not invest in an ML project if you have no plan to put it in production, except of course when doing pure research.

The difficulties in model deployment and management have given rise to a new, specialized role: the machine learning engineer. Machine learning engineers are closer to software engineers than typical data scientists, and as such, they are the ideal candidate to put models into production.

The goal of building a machine learning model is to solve a problem, and a machine learning model can only do so when it is in production and actively in use by consumers. As such, model deployment is as important as model building.

Venturebeat reports that 87% of data science projects never make it to production, while redapt claims it is 90%.

It was great for three things: motivation (people used what I built!)feedback (they sent me the bugs they encountered)accountability (from the moment I tweeted about this publicly, I really felt like that I’d have to finish this project… otherwise, it would be embarrassing)

Turning my Jupyter-compatible Python code into Flask-compatible Python code took dozens of hours. I ran into small bugs and errors

I had a proof-of-concept. It was great, because when you do a project, there’s always that small uncertainty that your idea is stupid and it can’t be done at all. But when you have a working prototype, all that insecurity is gone and it becomes much easier to go on.

The point is: the initial idea should be simple—because even that will turn out to be pretty complex.

The problem with hobby projects is that they don’t pay very well. At least, compared to real-life projects. But hobby projects have a lot of cool advantages that we tend to forget.

The user has a starting stack: 10,000 units of play money.In each round, she gets a deal. Something like “You have an x% probability to win y amount of money.”She can decide how much of her play money she wants to risk.And the goal is to win as much as she can in 50 rounds.

$ git log --grep "commit message search"

$ git log -S "deleted code" -p

$ git log -S "some code" -p app/models/user.rb

$ git log -G "REGEX HERE"

Find the entire history of a snippet of code with git log -S Include -p to see the diff as well as the commit messages Include --reverse to see the commit that introduced the code listed first Scope search to specific folders or files by including a path Search with a regular expression using git log -G Search commit messages using git log --grep

If you include the -p option (short for --patch) you get the full diff alongside the commit messages

Say you want to find the first commit that introduced a particular class, method or snippet of code. You can use the pickaxe combined with the --reverse option to get the commits in reverse-chronological order so the commit where the code first appears is listed at the top

git log -S (also known as “the pickaxe”) is how you take things to the next level. It lets you search for all commits that contain a given string:

git blame is too coarse: it reports against the whole line. If the most recent change isn’t related to the part of the line you’re interested, you’re out of luck. git blame is too shallow: it only reports a single change; the most recent one. The story of the particular piece of code you’re interested in may have evolved over several commits. git blame is too narrow: it only considers the file you are running blame against. The code you are interested in may also appear in other files, but to get the relevant commits on those you’ll need to run blame several times.

BlackArch is a Linux distribution designed for penetration testing and security research. You can think of it like Kali Linux, with the exception of being based on Arch Linux. Its official repositories contain more than +2500 various penetration testing tools, and hence it can be considered a very good option for anyone wishing to explore this field and try to hack their own phones/routers/devices during the quarantine time.

Redstar OS is a Fedora-based Linux distribution that was developed to empower all computers running in North Korea. Researchers from various independent teams verified that the distribution contains huge spying mechanisms and logging capability, beside some modified kernel modules to record everything done on the OS.

You can have more experiences in life by simply paying more attention. What you pay attention to is going to be your life. It's so basic that it hits me.

Everyone hates the feeling of having wasted their time doing nothing. But, even when I am doing many things in a day, when I am on autopilot, which is to say doing something for the sake of just completing them, I don't feel fulfilled. This makes total sense because the day wasn't filled with any intentionality. There was no mind.

Attention is like energy in that without it no work can be done, and in doing work it is dissipated. We create ourselves by how we invest this energy. Memories, thoughts, and feelings are all shaped by how we use it. And it is an energy under our control, to do with as we please; hence, attention is our most important tool in the task of improving the quality of experience.

Automation suggests that a sysadmin has invented a system to cause a computer to do something that would normally have to be done manually. In automation, the sysadmin has already made most of the decisions on what needs to be done, and all the computer must do is execute a "recipe" of tasks. Orchestration suggests that a sysadmin has set up a system to do something on its own based on a set of rules, parameters, and observations. In orchestration, the sysadmin knows the desired end result but leaves it up to the computer to decide what to do.

For instance, automation usually involves scripting, often in Bash or Python or similar, and it often suggests scheduling something to happen at either a precise time or upon a specific event. However, orchestration often begins with an application that's purpose-built for a set of tasks that may happen irregularly, on demand, or as a result of any number of trigger events, and the exact results may even depend on a variety of conditions.

Start to parse the HTML Fetch external resources Parse the CSS and build the CSSOM Execute the JavaScript Merge DOM and CSSOM to construct the render tree Calculate layout and paint

If we’re honest, we’ll admit that we forget 95% of what we learn in a class. What sticks? A scattered analogy or diagram.

Analogy – ? Diagram – Google Images Example – Khan Academy for practice problems Plain-English – Forums like /r/math or Math Overflow Technical – Wikipedia or MathWorld