CSW

does anyone use AMQP

This the specification of the Source object

Apache Airflow

DataFlows

WUI

creaate

processes

on bare metal

communication between pods

Should you place descriptions of code inside code comments or in text (paragraphs or lists) outside of the sample code? Note that readers who copy-and-paste a snippet gather not only the code but also any embedded comments. So, put any descriptions that belong in the pasted code into the code comments. By contrast, when you must explain a lengthy or tricky concept, you should typically place the text before the sample program.

When your readers are very experienced with a technology, don't explain what the code is doing, explain why the code is doing it.

According to research by Sung and Mayer (2012), providing any graphics—good or bad—makes readers like the document more; however, only instructive graphics help readers learn.

Most readers appreciate at least a brief introduction under each heading to provide some context. Avoid placing a level three heading immediately after a level two heading, as in the following example

Conversely, don't make paragraphs too short. If your document contains plenty of one-sentence paragraphs, your organization is faulty.

Long paragraphs are visually intimidating. Very long paragraphs form a dreaded "wall of text" that readers ignore. Readers generally welcome paragraphs containing three to five sentences, but will avoid paragraphs containing more than about seven sentences.

Avoid putting too much text into a table cell. If a table cell holds more than two sentences, ask yourself whether that information belongs in some other format.

If the list item is a sentence, use sentence capitalization and punctuation. Otherwise, do not use sentence capitalization and punctuation.

Consider starting all items in a numbered list with an imperative verb.

Sentences that start with There is or There are marry a generic noun to a generic verb. Generic weddings bore readers.

Many technical writers believe that the verb is the most important part of a sentence. Pick the right verb and the rest of the sentence will take care of itself.

Most readers mentally convert passive voice to active voice.

passive verb = form of be + past participle verb

In an active voice sentence, an actor acts on a target. That is, an active voice sentence follows this formula: Active Voice Sentence = actor + verb + target A passive voice sentence reverses the formula. That is, a passive voice sentence typically follows the following formula: Passive Voice Sentence = target + verb + actor

Use either of the following tactics to disambiguate this and that: Replace this or that with the appropriate noun. Place a noun immediately after this or that.

As a rule of thumb, if more than five words separate your noun from your pronoun, consider repeating the noun instead of using the pronoun.

See http://mathesaurus.sf.net/ for another MATLAB®/NumPy cross-reference.

from numpy import *

‘array’ or ‘matrix’? Which should I use?

In MATLAB®, arrays have pass-by-value semantics, with a lazy copy-on-write scheme to prevent actually creating copies until they are actually needed. Slice operations copy parts of the array. In NumPy arrays have pass-by-reference semantics. Slice operations are views into an array.

MATLAB® uses 1 (one) based indexing. The initial element of a sequence is found using a(1). See note INDEXING Python uses 0 (zero) based indexing. The initial element of a sequence is found using a[0].

In MATLAB®, the basic data type is a multidimensional array of double precision floating point numbers. Most expressions take such arrays and return such arrays. Operations on the 2-D instances of these arrays are designed to act more or less like matrix operations in linear algebra. In NumPy the basic type is a multidimensional array. Operations on these arrays in all dimensionalities including 2D are element-wise operations. One needs to use specific functions for linear algebra (though for matrix multiplication, one can use the @ operator in python 3.5 and above).

Some Key Differences

MATLAB® and NumPy/SciPy have a lot in common. But there are many differences. NumPy and SciPy were created to do numerical and scientific computing in the most natural way with Python, not to be MATLAB® clones.

NumPy supports a much greater variety of numerical types than Python does.

Who Else Uses NumPy?

Beware: matplotlib also has a function to build histograms (called hist, as in Matlab) that differs from the one in NumPy.

See linalg.py in numpy folder for more.

Linear Algebra¶ Work in progress. Basic linear algebra to be included here.

rg

The matrix product can be performed using the @ operator (in python >=3.5) or the dot function or method

elementwise

[20,30,40,50]

it’s very simple

The best and easiest way to do this is to use Pandas.

You can save a NumPy array as a plain text file like a .csv or .txt file with np.savetxt.

::

You can save it as “filename.npy” with: >>>>>> np.save('filename', a) You can use np.load() to reconstruct your array. >>>>>> b = np.load('filename.npy')

handle NumPy binary files with a .npy file extension, and a savez function that handles NumPy files with a .npz file extension.

You will, at some point, want to save your arrays to disk and load them back without having to re-run the code.

For example, this is the mean square error formula (a central formula used in supervised machine learning models that deal with regression): Implementing this formula is simple and straightforward in NumPy: What makes this work so well is that predictions and labels can contain one or a thousand values. They only need to be the same size.

When it comes to the data science ecosystem, Python and NumPy are built with the user in mind.

This section covers help(), ?, ??

The primary difference between the two is that the new array created using ravel() is actually a reference to the parent array (i.e., a “view”). This means that any changes to the new array will affect the parent array as well.

How to reverse an array

transpose your matrices

generate random numbers (actually, repeatable pseudo-random numbers)

You

NumPy

Views are an important NumPy concept!

shallow copy

ou can also stack two existing arrays, both vertically and horizontally.

slicing and indexing, np.vstack(), np.hstack(), np.hsplit(), .view(), copy()

a%2==0

You can visualize it this way:

This section covers ndarray.ndim, ndarray.size, ndarray.shape

, axis=0

This section covers np.sort(), np.concatenate()

np.arange(2, 9, 2)

Arrays and array operations are much more complicated than are captured here!

2D

You might occasionally hear an array referred to as a “ndarray,” which is shorthand for “N-dimensional array.” An N-dimensional array is simply an array with any number of dimensions. You might also hear 1-D, or one-dimensional array, 2-D, or two-dimensional array, and so on. The NumPy ndarray class is used to represent both matrices and vectors. A vector is an array with a single dimension (there’s no difference between row and column vectors), while a matrix refers to an array with two dimensions. For 3-D or higher dimensional arrays, the term tensor is also commonly used.

“0”

`0`

One way we can initialize NumPy arrays is from Python lists, using nested lists for two- or higher-dimensional data.

Why use NumPy?

it’s very easy to understand

We shorten

Installing NumPy

find more information

Need Discovery

NB

These are the Datopian Tech Leaders.

Needs Analysis

need or want to requirements.

Plan: planning work to deliver that solution. This includes breaking down a design into tasks, clarifying their dependencies and estimating these (i.e. a roadmap).

We emphasize that analysis is applicable both to large projects and to a single simple task. Just as test-driven development is worthwhile even for simple changes, so analyis will pay dividends even for small script or a minor change to a website.