cachew lets you cache function calls into an sqlite database on your disk in a matter of single decorator (similar to functools.lru_cache). The difference from functools.lru_cache is that cached data is persisted between program runs, so next time you call your function, it will only be a matter of reading from the cache.

supporting this field is extremely easy If you keep raw data, it's just a matter of adding a getter method to the Article class.

@property
def highlights(self) -> Sequence[Highlight]:
    default = [] # defensive to handle older export formats that had no annotations
    jsons = self.json.get('annotations', default)
    return list(map(Highlight, jsons))

query language doesn't necessarily mean a database. E.g. see pandas which is capable of what SQL is capable of, and even more convenient than SQL for our data exploration purposes.

By the way, what's up with Iterator everywhere?

# Import the Matcher from spacy.matcher import Matcher

In spaCy, attributes that return strings usually end with an underscore – attributes without the underscore return an ID.

“Python is now the new glue language,”

The nonlocal keyword is used to work with variables inside nested functions, where the variable should not belong to the inner function. Use the keyword nonlocal to declare that the variable is not local.

def myfunc1():
  x = "John"
  def myfunc2():
    nonlocal x
    x = "hello"
  myfunc2()
  return x

print(myfunc1())

We only need to use global keyword in a function if we want to do assignments / change them. global is not needed for printing and accessing.

# This function modifies global variable 's' 
def f(): 
    global s 
    print s 
    s = "Look for Geeksforgeeks Python Section"
    print s  

# Global Scope 
s = "Python is great!" 
f() 
print s

Since we have much faster CPUs now, numerical calculations are done in Python which is much slower than Fortran. So numerical calculations basically take the same amount of time as they did 20 years ago.

Sometimes, the best way to learn is to mimic others. Here are some great examples of projects that use documentation well:

Along with these tools, there are some additional tutorials, videos, and articles that can be useful when you are documenting your project

Documenting your code, especially large projects, can be daunting. Thankfully there are some tools out and references to get you started

Daniele Procida gave a wonderful PyCon 2017 talk and subsequent blog post about documenting Python projects. He mentions that all projects should have the following four major sections to help you focus your work:

The general layout of the project and its documentation should be as follows:

project_root/
│
├── project/  # Project source code
├── docs/
├── README
├── HOW_TO_CONTRIBUTE
├── CODE_OF_CONDUCT
├── examples.py

There are specific docstrings formats that can be used to help docstring parsers and users have a familiar and known format.

If you use argparse, then you can omit parameter-specific documentation, assuming it’s correctly been documented within the help parameter of the argparser.parser.add_argument function. It is recommended to use the __doc__ for the description parameter within argparse.ArgumentParser’s constructor.

Scripts are considered to be single file executables run from the console. Docstrings for scripts are placed at the top of the file and should be documented well enough for users to be able to have a sufficient understanding of how to use the script.

class constructor parameters should be documented within the __init__ class method docstring

Class method docstrings should contain the following: A brief description of what the method is and what it’s used for Any arguments (both required and optional) that are passed including keyword arguments Label any arguments that are considered optional or have a default value Any side effects that occur when executing the method Any exceptions that are raised Any restrictions on when the method can be called

Docstrings can be further broken up into three major categories: Class Docstrings: Class and class methods Package and Module Docstrings: Package, modules, and functions Script Docstrings: Script and functions

All multi-lined docstrings have the following parts: A one-line summary line A blank line proceeding the summary Any further elaboration for the docstring Another blank line

"""This is the summary line

This is the further elaboration of the docstring. Within this section,
you can elaborate further on details as appropriate for the situation.
Notice that the summary and the elaboration is separated by a blank new
line.

# Notice the blank line above. Code should continue on this line.

In all cases, the docstrings should use the triple-double quote (""") string format.

Docstring conventions are described within PEP 257. Their purpose is to provide your users with a brief overview of the object.

say_hello.__doc__ = "A simple function that says hello... Richie style"

def say_hello(name):
    print(f"Hello {name}, is it me you're looking for?")

say_hello.__doc__ = "A simple function that says hello... Richie style"

def say_hello(name):
    """A simple function that says hello... Richie style"""
    print(f"Hello {name}, is it me you're looking for?")

>>> help(say_hello)

Help on function say_hello in module __main__:

say_hello(name)
    A simple function that says hello... Richie style

Since everything in Python is an object, you can examine the directory of the object using the dir() command

Along with docstrings, Python also has the built-in function help() that prints out the objects docstring to the console.

Documenting your Python code is all centered on docstrings. These are built-in strings that, when configured correctly, can help your users and yourself with your project’s documentation.

From examining the type hinting, you can immediately tell that the function expects the input name to be of a type str, or string. You can also tell that the expected output of the function will be of a type str, or string, as well.

def hello_name(name: str) -> str:
    return(f"Hello {name}")

Well, why don't I create my own language', stealing my ideas from ABC

reducing the project size from something that took three years to complete to something I can do on my own, as a skunk works project in three months

Dutch programmer Guido van Rossum designed Python in 1991, naming it after the British television comedy Monty Python's Flying Circus because he was reading the show's scripts at the time.

Werkzeug provides a development server: a simple web server that you can run with a single command and almost no configuration. When you do flask run (or werkzeug.serving.run_simple()), this development server is what you are getting

I want to get the selected number of bins from the slider and pass that number into a python method and do some calculation/manipulation (return: “You have selected 30bins and I came from a Python Function”) inside of it then return some value back to my R Shiny dashboard and view that result in a text field.

x = [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]] arr = [i for j in x for i in j] print(arr) >>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

x = [[0, 1, 2, 3, 4],
 [5, 6, 7, 8, 9]]

arr = [i for j in x for i in j]
print(arr)
>>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

# Creating a 5x5 matrix arr = [[i for i in range(5)] for j in range(5)] arr >>> [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]

arr = [[i for i in range(5)] for j in range(5)]
arr
>>> [[0, 1, 2, 3, 4],
 [0, 1, 2, 3, 4],
 [0, 1, 2, 3, 4],
 [0, 1, 2, 3, 4],
 [0, 1, 2, 3, 4]]

arr = [i for i in range(10) if i % 2 == 0] print(arr) >>> [0, 2, 4, 6, 8] arr = ["Even" if i % 2 == 0 else "Odd" for i in range(10)] print(arr) >>> ['Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd']

arr = [i for i in range(10) if i % 2 == 0]
print(arr)
>>> [0, 2, 4, 6, 8]

arr = ["Even" if i % 2 == 0 else "Odd" for i in range(10)]
print(arr)
>>> ['Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd']

x = [2,45,21,45] y = {i:v for i,v in enumerate(x)} print(y) >>> {0: 2, 1: 45, 2: 21, 3: 45}

x = [2,45,21,45]
y = {i:v for i,v in enumerate(x)}
print(y)
>>> {0: 2, 1: 45, 2: 21, 3: 45}

The reason that Julia is fast (ten to 30 times faster than Python) is because it is compiled and not interpreted

Scala is ten times faster than Python

Vaex is an open-source DataFrame library which enables the visualisation, exploration, analysis and even machine learning on tabular datasets that are as large as your hard-drive. To do this, Vaex employs concepts such as memory mapping, efficient out-of-core algorithms and lazy evaluations.

HTTPretty

vectors here have length 7 instead of 6 because of the extra 0 element at the beginning. This is an inconsequential detail - Keras reserves index 0 and never assigns it to any word.

Here’s my preferred way of doing it, which uses Keras’s Tokenizer class

import operator s = sorted(s, key = operator.itemgetter(1, 2))

to reverse to only one attribute, you can sort twice: first by the secondary s = sorted(s, key = operator.itemgetter(2)) then by the primary s = sorted(s, key = operator.itemgetter(1), reverse=True)

Python code should be elegant, concise, and readable. It should be beautiful. The ultimate resource on how to write beautiful Python code is Style Guide for Python Code or PEP 8

what if an exception occurs while processing your file? Then my_file.close() is never executed. You can handle this with exception-handling syntax or with context managers

>>> with open('filename.csv', 'w') as my_file:
...     # do something with `my_file`

open a file and process it

>>> my_file = open('filename.csv', 'w')
>>> # do something with `my_file`

>>> my_file = open('filename.csv', 'w')
>>> # do something with `my_file and`
>>> my_file.close()

The author of the class probably begins the names with the underscore(s) to tell you, “don’t use it”

Python doesn’t have real private class members. However, there’s a convention that says that you shouldn’t access or modify the members beginning with the underscore (_) outside their instances. They are not guaranteed to preserve the existing behavior.

>>> class C:
...     def __init__(self, *args):
...         self.x, self._y, self.__z = args
... 
>>> c = C(1, 2, 4)

>>> c.x  # OK
1

>>> c._y  # Possible, but a bad practice!
2

>>> c.__z # Error!
Traceback (most recent call last):
File "", line 1, in 
AttributeError: 'C' object has no attribute '__z'
>>> c._C__z # Possible, but even worse!
4
>>>

it’s a good practice to use properties when you can and C++-like getters and setters when you have to

it’s often more elegant to define and use properties, especially in simple cases

>>> class C:
...     @property
...     def x(self):
...         return self.__x
...     @x.setter
...     def x(self, value):
...         self.__x = value

>>> c = C()
>>> c.x = 2
>>> c.x
2

Python allows defining getter and setter methods similarly as C++ and Java

>>> class C:
...     def get_x(self):
...         return self.__x
...     def set_x(self, value):
...         self.__x = value

>>> c = C()
>>> c.set_x(2)
>>> c.get_x()
2

You can keep away from that with some additional logic. One of the ways is this:

>>> def f(value, seq=[]):
...     seq.append(value)
...     return seq

>>> def f(value, seq=None):
...     if seq is None:
...         seq = []
...     seq.append(value)
...     return seq

>>> def f(value, seq=None):
...     if not seq:
...         seq = []
...     seq.append(value)
...     return seq

>>> f(value=2)
[2]
>>> f(value=4)
[4]
>>> f(value=8)
[8]
>>> f(value=16)
[16]

Python has a very flexible system of providing arguments to functions and methods. Optional arguments are a part of this offer. But be careful: you usually don’t want to use mutable optional arguments

>>> def f(value, seq=[]):
...     seq.append(value)
...     return seq

>>> f(value=2)
[2]

>>> f(value=4)
[2, 4]
>>> f(value=8)
[2, 4, 8]
>>> f(value=16)
[2, 4, 8, 16]

The Pythonic way is to exploit the fact that zero is interpreted as False in a Boolean context, while all other numbers are considered as True

>>> bool(0)
False
>>> bool(-1), bool(1), bool(20), bool(28.4)
(True, True, True, True)

>>> for item in x:
...     if item:
...         print(item)
... 
1
2
3
4

When you have numeric data, and you need to check if the numbers are equal to zero, you can but don’t have to use the comparison operators == and !=

>>> x = (1, 2, 0, 3, 0, 4)
>>> for item in x:
...     if item != 0:
...         print(item)
... 
1
2
3
4

Iterating over a dictionary yields its keys

>>> z = {'a': 0, 'b': 1}
>>> for k in z:
... print(k, z[k])
... 
a 0
b 1

>>> for k, v in z.items():
...     print(k, v)
... 
a 0
b 1

what if you want to iterate over two or more sequences? Of course, you can use the range again

>>> y = 'abcde'
>>> for i in range(len(x)):
...     print(x[i], y[i])
... 
1 a
2 b
4 c
8 d
16 e

>>> for item in zip(x, y):
...     print(item)
... 
(1, 'a')
(2, 'b')
(4, 'c')
(8, 'd')
(16, 'e')

>>> for x_item, y_item in zip(x, y):
...     print(x_item, y_item)
... 
1 a
2 b
4 c
8 d
16 e

Sometimes you need both the items from a sequence and the corresponding indices

>>> for i in range(len(x)):
...     print(i, x[i])
... 
0 1
1 2
2 4
3 8
4 16

>>> for i, item in enumerate(x):
...     print(i, item)
... 
0 1
1 2
2 4
3 8
4 16

But what if you want to iterate in the reversed order? Of course, the range is an option again

>>> for i in range(len(x)-1, -1, -1):
...     print(x[i])
... 
16
8
4
2
1

>>> for item in x[::-1]:
...     print(item)
... 
16
8
4
2
1

>>> for item in reversed(x):
...     print(item)
... 
16
8
4
2
1

In almost all cases, you can use the range to get an iterator that yields integers

>>> x = [1, 2, 4, 8, 16]
>>> for i in range(len(x)):
...     print(x[i])
... 
1
2
4
8
16

>>> for item in x:
...     print(item)
... 
1
2
4
8
16

It’s possible to check whether a variable refers to it with the comparison operators == and !=

>>> x, y = 2, None
>>> x == None
False
>>> y == None
True
>>> x != None
True
>>> y != None
False

>>> x is None
False
>>> y is None
True
>>> x is not None
True
>>> y is not None
False

None is a special and unique object in Python. It has a similar purpose, like null in C-like languages

Python also supports chained assignments. So, if you want to assign the same value to multiple variables, you can do it in a straightforward way

>>> x = 2
>>> y = 2
>>> z = 2

>>> x, y, z = 2, 2, 2

>>> x = y = z = 2
>>> x, y, z
(2, 2, 2)

Python allows you to chain the comparison operations. So, you don’t have to use and to check if two or more comparisons are True

>>> x = 4
>>> x >= 2 and x <= 8
True

>>> 2 <= x <= 8
True
>>> 2 <= x <= 3
False

Unpacking can be used for the assignment to multiple variables in more complex cases

>>> x = (1, 2, 4, 8, 16)
>>> a = x[0]
>>> b = x[1]
>>> c = x[2]
>>> d = x[3]
>>> e = x[4]
>>> a, b, c, d, e
(1, 2, 4, 8, 16)

>>> a, b, c, d, e = x
>>> a, b, c, d, e
(1, 2, 4, 8, 16)

>>> a, *y, e = x
>>> a, e, y
(1, 16, [2, 4, 8])

the most concise and elegant variables swap

>>> a, b = b, a
>>> a
'my-string'
>>> b
2

You can use unpacking to assign values to your variables

>>> a, b = 2, 'my-string'
>>> a
2
>>> b
'my-string'

following the rules called The Zen of Python or PEP 20

One of my favorite languages that I have had the pleasure to use is Python. Python supports named parameters without any trickery.... Since I started using Python (some time ago) everything became easier. I believe that every language should support named parameters, but that just isn't the case.

this article merely scratches the surface. If you want to dig deeper into concurrency in Python, there is an excellent talk titled Thinking about Concurrency by Raymond Hettinger on the subject. Make sure to also check out the slides whilst you’re at it

Introducing multiprocessing now is a cinch; I just replace ThreadPoolExecutor with ProcessPoolExecutor in the previous listing

As the name suggests, multiprocessing spawns processes, while multithreading spawns threads. In Python, one process can run several threads. Each process has its proper Python interpreter and its proper GIL. As a result, starting a process is a heftier and more time-consuming undertaking than starting a thread.

First of all, write a script that carries out the task in a sequential fashion. Secondly, transform the script so that it carries out the task using the map function. Lastly, replace map with a neat function from the concurrent.futures module

Python standard library makes it fairly easy to create threads and processes

Python is a poor choice for concurrent programming. A principal reason for this is the ‘Global Interpreter Lock’ or GIL. The GIL ensures that only one thread accesses Python objects at a time, effectively preventing Python from being able to distribute threads onto several CPUs by default

Indicate number of NA values placed in non-numeric columns.

Filled 3 NA values in column name

Tokenization took: 0.01 ms
Type conversion took: 0.70 ms
Parser memory cleanup took: 0.01 ms

def getAnswer(answerNumber):

expression

return 3 + 5
# 8

It’s just that it often makes sense to write code in the order JOIN / WHERE / GROUP BY / HAVING. (I’ll often put a WHERE first to improve performance though, and I think most database engines will also do a WHERE first in practice)

Passlib mitigates this by limiting the maximum password size to 4k by default.

Which works fine, but make you use lambda. You could rather use the operator module:import operator mylist = list(zip(range(40, 240), range(-100, 100))) sorted(mylist, key=operator.itemgetter(1))

Good Integration Practices

py27-django{18,19,110,111,111tip},

the __ methods allow us to interact with core concepts of the python language. You can see them also as a mechanism of implementing behaviours, interface methods.

Dunder or magic method, are methods that start and end with double _ like __init__ or __str__. This kind of methods are the mechanism we use to interact directly with python's data model

use pyenv. With it, you will be able to have any version you want at your disposal, very easy.

Another common component to profile is the memory usage. The purpose is to find memory leaks and optimize the memory usage in your Python programs

The profile module gives similar results with similar commands. Typically, you switch to profile if cProfile isn’t available

Line profiling, as the name suggests, means to profile your Python code line by line

If a method has an acceptable speed but is so frequently called that it becomes a huge time sink, you would want to know this from your profiler

method profiling tool like cProfile (which is available in the Python language), the timing metrics for methods can show you statistics, such as the number of calls (shown as ncalls), total time spent in the function (tottime), time per call (tottime/ncalls and shown as percall), cumulative time spent in a function (cumtime), and cumulative time per call (quotient of cumtime over the number of primitive calls and shown as percall after cumtime)

what parts of the software do we profile (measure its performance metrics)

its purpose is to dump Python tracebacks explicitly on a fault, after a timeout, or on a user signal

The purpose of trace module is to “monitor which statements and functions are executed as a program runs to produce coverage and call-graph information

trace and faulthandler modules cover basic tracing

tracing is a special use case of logging in order to record information about a program’s execution

intern

ascii letters, digits or underscores

sequences generated through peephole optimization are discarded if their length is superior to 20

'a' * 21 # remains in the byte code where as
'a' * 20 # gets converted to 'aaaaaaaaaaaaaaaaaaaa'

string subclasses cannot be interned

class NewString(str):
   pass
assert NewString('f') is 'f'

f = 'f'
assert f is 'f'

strings can be compared by a O(1) pointer comparison instead of a O(n) byte-per-byte comparison

// compare pointers
if self._value is value:
   return True

// compare values
for i, v in enumerate(self._value):
   if v != value[i]:
      return False
return True

hetio

On 10 April, astrophysicists announced that they had captured the first ever image of a black hole. This was exhilarating news, but none of the giddy headlines mentioned that the image would have been impossible without open-source software. The image was created using Matplotlib, a Python library for graphing data, as well as other components of the open-source Python ecosystem. Just five days later, the US National Science Foundation (NSF) rejected a grant proposal to support that ecosystem, saying that the software lacked sufficient impact.

while number:

get_current_request()

Corporate training Introduction to Python Advanced Python Python for non-programmers Data science and machine learning in Python Python for system administrators Python Practice Workshop Regular expressions Introduction to Git Online training Weekly Python Exercise NumPy Intro Python: Fundamentals Object-oriented Python Comprehending Comprehensions Understanding and mastering Git Python Workout Practice Makes Regexp Free e-mail courses Regular expressions crash course Boolean indexing in NumPy and Pandas Variable scoping in Python Working with files in Python Teach programming better Blog About Reuven Home → Blog →Python →Why do Python lists let you += a tuple, when you can’t + a tuple? 1 Why do Python lists let you += a tuple, when you can’t + a tuple? Let’s say you have a list in Python: >>> mylist = [10, 20, 30] You want to add something to that list. The most standard way to do this is with the “append” method, which adds its argument to the end of the list: >>> mylist.append(40) >>> print(mylist) [10, 20, 30, 40] But what if you want to add multiple items to a list? If you’re new to Python, then you might think that you can and should use a “for” loop. For example: >>> mylist = [10, 20, 30] >>> new_items = [40, 50, 60] >>> for one_item in new_items: mylist.append(one_item) >>> print(mylist) [10, 20, 30, 40, 50, 60] Great, right? But it turns out that there is a smarter and faster way to do this. You can use the += operator. This operator, which invokes the “iadd” (“inplace add”) method on the object to its left, effectively does what we did above, but in much less code: >>> mylist = [10, 20, 30] >>> new_items = [40, 50, 60] >>> mylist += new_items >>> print(mylist) [10, 20, 30, 40, 50, 60] It’s not a huge surprise that += can do this. After all, we normally expect += to add and assign to the variable on its left; it works with numbers and strings, as well as other types. And we know that we can use the + operator on lists, too: >>> [1, 2, 3] + [4, 5, 6] [1, 2, 3, 4, 5, 6] Can we join a list and a tuple? Let’s check: >>> mylist = [10, 20, 30] >>> t = (40, 50, 60) >>> mylist + t Traceback (most recent call last): File "", line 1, in TypeError: can only concatenate list (not "tuple") to list In other words: No. Trying to add a list and a tuple, even if we’re not affecting either, results in the above error. Which is why it’s so surprising to many of my students that the following does work: >>> mylist = [10, 20, 30] >>> t = (40, 50, 60) >>> mylist += t >>> mylist [10, 20, 30, 40, 50, 60] That’s right: Adding a list to a tuple with + doesn’t work. But if we use +=, it does. What gives? It’s common, when teaching Python, to say that x += 5 is basically a rewrite of x = x + 5 And in the majority of cases, that’s actually true. But it’s not always true. Consider: When you say “x + y” in Python, the “+” operator is translated into a method call. Behind the scenes, no matter what “x” and “y” are, the expression is translated into: x.__add__(y) The “__add__” magic method is what’s invoked on an object when it is added to another object. The object on the right-hand side of the “+” is passed as an argument to the method, while the object on the left-hand side is the recipient of the method call. That’s why, if you want your own objects to handle the “+” operator, you need to define the “__add__” method in your class definition. Do that, and things work just fine. And thus, when we say “x = x + 5”, this is turned into x = x.__add__(5) Meaning: First invoke the method, and then assign it back to the variable “x”. In this case, “x” isn’t changing; rather, the variable is now referencing a new object. Now consider the “+=” operator: It’s translated by Python into “__iadd__”, short for “inplace add.” Notice the slightly different syntax that we use here: x += y is translated into x.__iadd__(y) Did you see the difference between __add__ and __iadd__? The latter executes the assignment all by itself, internally. You don’t have to capture its output and assign it back to x. It turns out that the implementation of list.__iadd__ takes the second (right-hand side) argument and adds it, one element at a time, to the list. It does this internally, so that you don’t need to execute any assignment after. The second argument to “+=” must be iterable; if you say mylist += 5 you will get an error, saying that integers are not iterable. But if you put a string, list, tuple, or any other iterable type on the right-hand side, “+=” will execute a “for” loop on that object, adding each of its elements, one at a time, to the list. In other words: When you use + on a list, then the right-hand object must be a list. But when you use +=, then any iterable type is acceptable: >>> mylist = [10, 20, 30] >>> mylist += [40, 50] # list >>> mylist [10, 20, 30, 40, 50] >>> mylist += (60, 70) # tuple >>> mylist [10, 20, 30, 40, 50, 60, 70] >>> mylist += 'abc' # string >>> mylist [10, 20, 30, 40, 50, 60, 70, 'a', 'b', 'c'] >>> mylist += {'x':1, 'y':2, 'z':3} # dict! >>> mylist [10, 20, 30, 40, 50, 60, 70, 'a', 'b', 'c', 'x', 'y', 'z'] Does this work with other types? Not really. For example: >>> t = (10, 20, 30) >>> t += [40, 50] Traceback (most recent call last): File "", line 1, in TypeError: can only concatenate tuple (not "list") to tuple What happened here? Let’s check the definition of tuple.__iadd__ to find out: >>> help(tuple.__iadd__) Traceback (most recent call last): File "", line 1, in AttributeError: type object 'tuple' has no attribute '__iadd__' Wait a second: There is no “__iadd__” method for tuples? If so, then how can “+=” work at all? Because Python tries to be smart in such cases: If the object implements “__iadd__”, then the “+=” operator invokes it. But if the object lacks an “__iadd__” implementation, then Python does what we all guess it normally does — namely, invoke “__add__”, and then assign the results back to the variable. For example: >>> class Foo(object): def __init__(self, x): self.x = x def __add__(self, other): print("In __add__") return Foo(self.x + other.x) >>> f1 = Foo(10) >>> f2 = Foo(20) >>> f1 += f2 In __add__ >>> vars(f1) {'x': 30} In other words, Python notices that our Foo class lacks an implementation of “__iadd__”, and substitutes “__add__” for it, assigning its result (a new instance of Foo) to the original variable.

#!/usr/bin/env python3 import configparser import os import sys from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.keys import Keys from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By config = configparser.ConfigParser() config.read("config.ini") chrome_options = Options() chrome_options.add_argument("--headless") driver = webdriver.Chrome(executable_path=os.path.abspath("/usr/local/bin/chromedriver"), chrome_options=chrome_options) driver.get("https://fastmail.fm") timeout = 120 try: element_present = EC.presence_of_element_located((By.NAME, 'username')) WebDriverWait(driver, timeout).until(element_present) # Send login information user = driver.find_element_by_name("username") passwd = driver.find_element_by_name("password") user.send_keys(config["default"]["user"]) passwd.send_keys(config["default"]["pass"]) driver.find_element_by_class_name("v-Button").click() print("Logged in") # wait for login to complete element_present = EC.presence_of_element_located((By.CLASS_NAME, 'v-MainNavToolbar')) WebDriverWait(driver, timeout).until(element_present) # click settings menu to make elements visible driver.find_element_by_class_name("v-MainNavToolbar").click() # And follow to settings page driver.find_element_by_link_text("Settings").click() # Wait for settings page to render, oh Javascript element_present = EC.presence_of_element_located((By.LINK_TEXT, 'Rules')) WebDriverWait(driver, timeout).until(element_present) # Click on Rules link driver.find_element_by_link_text("Rules").click() # Click on edit custom sieve code element_present = EC.presence_of_element_located((By.LINK_TEXT, 'Edit custom sieve code')) WebDriverWait(driver, timeout).until(element_present) driver.find_element_by_link_text("Edit custom sieve code").click() print("Editing") # This is super unstable, I hate that we have to go by webid element_present = EC.presence_of_element_located((By.CLASS_NAME, 'v-EditSieve-rules')) WebDriverWait(driver, timeout).until(element_present) print("Find form") elements = driver.find_elements_by_css_selector("textarea.v-Text-input") element = elements[-1] # Find the submit button elements = driver.find_elements_by_css_selector("button") for e in elements: if "Save" in e.text: submit = e print("Found form") # And replace the contents element.clear() with open("rules.txt") as f: element.send_keys(f.read()) # This is the Save button print("Submitted!") submit.click() except TimeoutException as e: print(e) print("Timed out waiting for page to load") sys.exit(0) print("Done!")

Python is currently the most popular language used by developers working on machine learning projects, according to GitHub’s recent Octoverse report

The Ultimate Guide to Python Type Checking

Type hints cheat sheet (Python 3)¶ This document is a quick cheat sheet showing how the PEP 484 type annotation notation represents various common types in Python 3.

MIT “MATLAB is the language used by virtually every team in the world that designs gravitational wave detectors… I look forward to exploring the data from each new detection in MATLAB.” Matthew Evans, Assistant Professor of Physics

KNIME includes Python in various processing nodes for data processing, model learning and prediction, and the generation of visualizations.

大概就是Machine这个类里面维护比较重要的三个元素，States={}，Events={}, models= []

All you can do is send a message (AYCDISAM) = Actors model - there is no direct manipulation of objects, only communication with (or invocation of) them. The presence of fields in Java violates this.

Matplotlib different size subplots

matplotlib.pyplot.subplots_adjust

if you need to pull out these rows and examine them

With over 6 million users, the open source Anaconda Distribution is the fastest and easiest way to do Python and R data science and machine learning on Linux, Windows and Mac OS X. It's the industry standard for developing, testing and training on a single machine

如果不在同一个package中，例如我们希望在module_21.py中调用module_11.py中的FuncA

from module_11包名.module_11 import funcA

　　这两个函数都接收一个分割字符串作为参数，将目标字符串分割为两个部分，返回一个三元元组（head,sep,tail），包含分割符。细微区别在于前者从目标字符串的末尾也就是右边开始搜索分割符。

>>> "django.core.app".partition('.')
('django', '.', 'core.app')
>>> "django.core.app".rpartition('.')
('django.core', '.', 'app')

argarse.ArgumentParser.parse_known_args()解析

import argparse  
parser = argparse.ArgumentParser()  
parser.add_argument(  
    '--flag_int',  
    type=float,  
    default=0.01,  
    help='flag_int.'  
)  
FLAGS, unparsed = parser.parse_known_args()  
print(FLAGS)  
print(unparsed)

# python test.py --flag_int 1 --flag_bool True --flag_string 'haha, I get it' --flag_float 0.2

Namespace(flag_int=1.0)
['--flag_bool', 'True', '--flag_string', 'haha, I get it', '--flag_float', '0.2']

one level is chosen as the “reference”, and its mean behaviour is represented by the intercept. Each column of the resulting matrix represents the difference between the mean of one level and this reference level

Iterator Protocol

Classes allow us to create a custom type of object -- that is, an object with its own behaviors and its own ways of storing data.

Packages provide for accessing variables within multiple files.

A package is a directory of files that work together as a Python application or library module.

Recursion features three elements

*args and **kwargs to capture all arguments

object: a data value of a particular type variable: a name bound to an object

All programs named test_something.py that have functions named test_something()

unit test: testing individual units (function/method) integration test: testing multiple units together regression test: testing to see if changes have introduced errors end-to-end test: testing the entire program

pandas is a Python module used for manipulation and analysis of tabular data.

pandas official documentation

JupyterHub, a multi-user Hub, spawns, manages, and proxies multiple instances of the single-user Jupyter notebook server.

Turn a GitHub repo into a collection of interactive notebooks

Host, run, and code Python in the cloud!

GeoPandas

Folium

What is music21?

How simple is music21 to use?

At every turn, IPython chose the way that was more inclusive, to the point where it’s no longer called “IPython”: The project rebranded itself as “Jupyter” in 2014 to recognize the fact that it was no longer just for Python.