intern

ascii letters, digits or underscores

sequences generated through peephole optimization are discarded if their length is superior to 20

'a' * 21 # remains in the byte code where as
'a' * 20 # gets converted to 'aaaaaaaaaaaaaaaaaaaa'

string subclasses cannot be interned

class NewString(str):
   pass
assert NewString('f') is 'f'

f = 'f'
assert f is 'f'

strings can be compared by a O(1) pointer comparison instead of a O(n) byte-per-byte comparison

// compare pointers
if self._value is value:
   return True

// compare values
for i, v in enumerate(self._value):
   if v != value[i]:
      return False
return True

hetio

On 10 April, astrophysicists announced that they had captured the first ever image of a black hole. This was exhilarating news, but none of the giddy headlines mentioned that the image would have been impossible without open-source software. The image was created using Matplotlib, a Python library for graphing data, as well as other components of the open-source Python ecosystem. Just five days later, the US National Science Foundation (NSF) rejected a grant proposal to support that ecosystem, saying that the software lacked sufficient impact.

while number:

get_current_request()

Corporate training Introduction to Python Advanced Python Python for non-programmers Data science and machine learning in Python Python for system administrators Python Practice Workshop Regular expressions Introduction to Git Online training Weekly Python Exercise NumPy Intro Python: Fundamentals Object-oriented Python Comprehending Comprehensions Understanding and mastering Git Python Workout Practice Makes Regexp Free e-mail courses Regular expressions crash course Boolean indexing in NumPy and Pandas Variable scoping in Python Working with files in Python Teach programming better Blog About Reuven Home → Blog →Python →Why do Python lists let you += a tuple, when you can’t + a tuple? 1 Why do Python lists let you += a tuple, when you can’t + a tuple? Let’s say you have a list in Python: >>> mylist = [10, 20, 30] You want to add something to that list. The most standard way to do this is with the “append” method, which adds its argument to the end of the list: >>> mylist.append(40) >>> print(mylist) [10, 20, 30, 40] But what if you want to add multiple items to a list? If you’re new to Python, then you might think that you can and should use a “for” loop. For example: >>> mylist = [10, 20, 30] >>> new_items = [40, 50, 60] >>> for one_item in new_items: mylist.append(one_item) >>> print(mylist) [10, 20, 30, 40, 50, 60] Great, right? But it turns out that there is a smarter and faster way to do this. You can use the += operator. This operator, which invokes the “iadd” (“inplace add”) method on the object to its left, effectively does what we did above, but in much less code: >>> mylist = [10, 20, 30] >>> new_items = [40, 50, 60] >>> mylist += new_items >>> print(mylist) [10, 20, 30, 40, 50, 60] It’s not a huge surprise that += can do this. After all, we normally expect += to add and assign to the variable on its left; it works with numbers and strings, as well as other types. And we know that we can use the + operator on lists, too: >>> [1, 2, 3] + [4, 5, 6] [1, 2, 3, 4, 5, 6] Can we join a list and a tuple? Let’s check: >>> mylist = [10, 20, 30] >>> t = (40, 50, 60) >>> mylist + t Traceback (most recent call last): File "", line 1, in TypeError: can only concatenate list (not "tuple") to list In other words: No. Trying to add a list and a tuple, even if we’re not affecting either, results in the above error. Which is why it’s so surprising to many of my students that the following does work: >>> mylist = [10, 20, 30] >>> t = (40, 50, 60) >>> mylist += t >>> mylist [10, 20, 30, 40, 50, 60] That’s right: Adding a list to a tuple with + doesn’t work. But if we use +=, it does. What gives? It’s common, when teaching Python, to say that x += 5 is basically a rewrite of x = x + 5 And in the majority of cases, that’s actually true. But it’s not always true. Consider: When you say “x + y” in Python, the “+” operator is translated into a method call. Behind the scenes, no matter what “x” and “y” are, the expression is translated into: x.__add__(y) The “__add__” magic method is what’s invoked on an object when it is added to another object. The object on the right-hand side of the “+” is passed as an argument to the method, while the object on the left-hand side is the recipient of the method call. That’s why, if you want your own objects to handle the “+” operator, you need to define the “__add__” method in your class definition. Do that, and things work just fine. And thus, when we say “x = x + 5”, this is turned into x = x.__add__(5) Meaning: First invoke the method, and then assign it back to the variable “x”. In this case, “x” isn’t changing; rather, the variable is now referencing a new object. Now consider the “+=” operator: It’s translated by Python into “__iadd__”, short for “inplace add.” Notice the slightly different syntax that we use here: x += y is translated into x.__iadd__(y) Did you see the difference between __add__ and __iadd__? The latter executes the assignment all by itself, internally. You don’t have to capture its output and assign it back to x. It turns out that the implementation of list.__iadd__ takes the second (right-hand side) argument and adds it, one element at a time, to the list. It does this internally, so that you don’t need to execute any assignment after. The second argument to “+=” must be iterable; if you say mylist += 5 you will get an error, saying that integers are not iterable. But if you put a string, list, tuple, or any other iterable type on the right-hand side, “+=” will execute a “for” loop on that object, adding each of its elements, one at a time, to the list. In other words: When you use + on a list, then the right-hand object must be a list. But when you use +=, then any iterable type is acceptable: >>> mylist = [10, 20, 30] >>> mylist += [40, 50] # list >>> mylist [10, 20, 30, 40, 50] >>> mylist += (60, 70) # tuple >>> mylist [10, 20, 30, 40, 50, 60, 70] >>> mylist += 'abc' # string >>> mylist [10, 20, 30, 40, 50, 60, 70, 'a', 'b', 'c'] >>> mylist += {'x':1, 'y':2, 'z':3} # dict! >>> mylist [10, 20, 30, 40, 50, 60, 70, 'a', 'b', 'c', 'x', 'y', 'z'] Does this work with other types? Not really. For example: >>> t = (10, 20, 30) >>> t += [40, 50] Traceback (most recent call last): File "", line 1, in TypeError: can only concatenate tuple (not "list") to tuple What happened here? Let’s check the definition of tuple.__iadd__ to find out: >>> help(tuple.__iadd__) Traceback (most recent call last): File "", line 1, in AttributeError: type object 'tuple' has no attribute '__iadd__' Wait a second: There is no “__iadd__” method for tuples? If so, then how can “+=” work at all? Because Python tries to be smart in such cases: If the object implements “__iadd__”, then the “+=” operator invokes it. But if the object lacks an “__iadd__” implementation, then Python does what we all guess it normally does — namely, invoke “__add__”, and then assign the results back to the variable. For example: >>> class Foo(object): def __init__(self, x): self.x = x def __add__(self, other): print("In __add__") return Foo(self.x + other.x) >>> f1 = Foo(10) >>> f2 = Foo(20) >>> f1 += f2 In __add__ >>> vars(f1) {'x': 30} In other words, Python notices that our Foo class lacks an implementation of “__iadd__”, and substitutes “__add__” for it, assigning its result (a new instance of Foo) to the original variable.

#!/usr/bin/env python3 import configparser import os import sys from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.keys import Keys from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By config = configparser.ConfigParser() config.read("config.ini") chrome_options = Options() chrome_options.add_argument("--headless") driver = webdriver.Chrome(executable_path=os.path.abspath("/usr/local/bin/chromedriver"), chrome_options=chrome_options) driver.get("https://fastmail.fm") timeout = 120 try: element_present = EC.presence_of_element_located((By.NAME, 'username')) WebDriverWait(driver, timeout).until(element_present) # Send login information user = driver.find_element_by_name("username") passwd = driver.find_element_by_name("password") user.send_keys(config["default"]["user"]) passwd.send_keys(config["default"]["pass"]) driver.find_element_by_class_name("v-Button").click() print("Logged in") # wait for login to complete element_present = EC.presence_of_element_located((By.CLASS_NAME, 'v-MainNavToolbar')) WebDriverWait(driver, timeout).until(element_present) # click settings menu to make elements visible driver.find_element_by_class_name("v-MainNavToolbar").click() # And follow to settings page driver.find_element_by_link_text("Settings").click() # Wait for settings page to render, oh Javascript element_present = EC.presence_of_element_located((By.LINK_TEXT, 'Rules')) WebDriverWait(driver, timeout).until(element_present) # Click on Rules link driver.find_element_by_link_text("Rules").click() # Click on edit custom sieve code element_present = EC.presence_of_element_located((By.LINK_TEXT, 'Edit custom sieve code')) WebDriverWait(driver, timeout).until(element_present) driver.find_element_by_link_text("Edit custom sieve code").click() print("Editing") # This is super unstable, I hate that we have to go by webid element_present = EC.presence_of_element_located((By.CLASS_NAME, 'v-EditSieve-rules')) WebDriverWait(driver, timeout).until(element_present) print("Find form") elements = driver.find_elements_by_css_selector("textarea.v-Text-input") element = elements[-1] # Find the submit button elements = driver.find_elements_by_css_selector("button") for e in elements: if "Save" in e.text: submit = e print("Found form") # And replace the contents element.clear() with open("rules.txt") as f: element.send_keys(f.read()) # This is the Save button print("Submitted!") submit.click() except TimeoutException as e: print(e) print("Timed out waiting for page to load") sys.exit(0) print("Done!")

Python is currently the most popular language used by developers working on machine learning projects, according to GitHub’s recent Octoverse report

The Ultimate Guide to Python Type Checking

Type hints cheat sheet (Python 3)¶ This document is a quick cheat sheet showing how the PEP 484 type annotation notation represents various common types in Python 3.

MIT “MATLAB is the language used by virtually every team in the world that designs gravitational wave detectors… I look forward to exploring the data from each new detection in MATLAB.” Matthew Evans, Assistant Professor of Physics

KNIME includes Python in various processing nodes for data processing, model learning and prediction, and the generation of visualizations.

大概就是Machine这个类里面维护比较重要的三个元素，States={}，Events={}, models= []

All you can do is send a message (AYCDISAM) = Actors model - there is no direct manipulation of objects, only communication with (or invocation of) them. The presence of fields in Java violates this.

Matplotlib different size subplots

matplotlib.pyplot.subplots_adjust

if you need to pull out these rows and examine them

With over 6 million users, the open source Anaconda Distribution is the fastest and easiest way to do Python and R data science and machine learning on Linux, Windows and Mac OS X. It's the industry standard for developing, testing and training on a single machine

如果不在同一个package中，例如我们希望在module_21.py中调用module_11.py中的FuncA

from module_11包名.module_11 import funcA

　　这两个函数都接收一个分割字符串作为参数，将目标字符串分割为两个部分，返回一个三元元组（head,sep,tail），包含分割符。细微区别在于前者从目标字符串的末尾也就是右边开始搜索分割符。

>>> "django.core.app".partition('.')
('django', '.', 'core.app')
>>> "django.core.app".rpartition('.')
('django.core', '.', 'app')

argarse.ArgumentParser.parse_known_args()解析

import argparse  
parser = argparse.ArgumentParser()  
parser.add_argument(  
    '--flag_int',  
    type=float,  
    default=0.01,  
    help='flag_int.'  
)  
FLAGS, unparsed = parser.parse_known_args()  
print(FLAGS)  
print(unparsed)

# python test.py --flag_int 1 --flag_bool True --flag_string 'haha, I get it' --flag_float 0.2

Namespace(flag_int=1.0)
['--flag_bool', 'True', '--flag_string', 'haha, I get it', '--flag_float', '0.2']

one level is chosen as the “reference”, and its mean behaviour is represented by the intercept. Each column of the resulting matrix represents the difference between the mean of one level and this reference level

Iterator Protocol

Classes allow us to create a custom type of object -- that is, an object with its own behaviors and its own ways of storing data.

Packages provide for accessing variables within multiple files.

A package is a directory of files that work together as a Python application or library module.

Recursion features three elements

*args and **kwargs to capture all arguments

object: a data value of a particular type variable: a name bound to an object

All programs named test_something.py that have functions named test_something()

unit test: testing individual units (function/method) integration test: testing multiple units together regression test: testing to see if changes have introduced errors end-to-end test: testing the entire program

pandas is a Python module used for manipulation and analysis of tabular data.

pandas official documentation

JupyterHub, a multi-user Hub, spawns, manages, and proxies multiple instances of the single-user Jupyter notebook server.

Turn a GitHub repo into a collection of interactive notebooks

Host, run, and code Python in the cloud!

GeoPandas

Folium

What is music21?

How simple is music21 to use?

At every turn, IPython chose the way that was more inclusive, to the point where it’s no longer called “IPython”: The project rebranded itself as “Jupyter” in 2014 to recognize the fact that it was no longer just for Python.

At some point in the following decade json-head.appspot.com stopped working. Today I’m bringing it back, mainly as an excuse to try out the combination of Python 3.5 async, the Sanic microframework and Zeit’s brilliant Now deployment platform.

c = np.full((2,2), 7) # Create a constant array print(c)

new to Python

However, unlike an ordinary loop, yield from allows subgenerators to receive sent and thrown values directly from the calling scope, and return a final value to the outer generator

Spectral Python (SPy) is a pure Python module for processing hyperspectral image data. It has functions for reading, displaying, manipulating, and classifying hyperspectral imagery. It can be used interactively from the Python command prompt or via Python scripts

PostgreSQL連接Python

Python 读写 txt 文本文件的操作方法全解析

Learn Data Science Online

Runestone Interactive

Shannon held up a BBC micro:bit board, which runs MicroPython and has been given to students in the UK, and noted that it only has 16KB of memory.

Calling from Python As of 10.10, AppleScript can use Cocoa frameworks in any Script Editor-based script. This makes it easy for Python and other languages that have Objective-C bridges to call AppleScript directly, without having to package everything as an Xcode app. Stick one or more compiled .scpt files containing AppleScript-ObjC "classes" into a folder, e.g.: use framework "Foundation" use scripting additions script MyASClass property parent : class "NSObject" -- on test() activate display dialog "Hello from AppleScript!" end test end script then just use it from Python like this: from Foundation import NSBundle, NSClassFromString import AppleScriptObjC NSBundle.alloc().initWithPath_(FOLDERPATH).loadAppleScriptObjectiveCScripts() MyASClass = NSClassFromString(u"MyASClass") # get the ASOC class... MyASClass.alloc().init().test() # ...then instantiate and call it

1608 The essay "Rhetoric" that is reprinted here comes from Doing What Comes Nat-urally.

Prophet : Facebook에서 오픈 소스로 공개한 시계열 데이터의 예측 도구로 R과 Python으로 작성되었다.

conciliatory

However, remember that Click is Unicode-based, so the string will always be a Unicode value

In Python, as well as in any other object-oriented programming language, we define a class to be a description of what the data look like (the state) and what the data can do (the behavior). Classes are analogous to abstract data types because a user of a class only sees the state and behavior of a data item. Data items are called objects in the object-oriented paradigm. An object is an instance of a class.

In that case, you need an empty placeholder list that you can append items to individually.

Mastodon.py Documentation

The higher the abstraction, the more detail is lost

Remember that a slicing tuple can always be constructed as obj and used in the x[obj] notation. Slice objects can be used in the construction in place of the [start:stop:step] notation. For example, x[1:10:5,::-1] can also be implemented as obj = (slice(1,10,5), slice(None,None,-1)); x[obj] . This can be useful for constructing generic code that works on arrays of arbitrary dimension.

That’s probably the Dropbox server. Two million lines of code and counting, and it serves hundreds of millions of users.

I want Python to be more effective for large projects, without losing sight of its use for small projects and teaching. It’s quite a challenge; my current hope lies in PEP 484 and mypy, an approach to optional static typing (a.k.a. gradual typing). It’s super exciting. There are also other things happening in the community that make Python faster.

The rule of thumb is to avoid using more than two expressions in a list comprehension. This could be two conditions, two loops, or one condition and one loop. As soon as it gets more complicated than that, you should use normal if and for statements and write a helper function

Dictionaries and sets have their own equivalents of list comprehen-sions.

d = { "foo": "bar", "baz": "qux" }
r = {val: key for key, val in d.items()}
#=> { "bar": "foo", "qux": "baz" }

Unlike map, list comprehensions let you easily filter items from the input list, removing corresponding outputs from the result.

Beware that indexing a list by a negative variable is one of the few situations in which you can get surprising results from slicing. For example, the expression so melist[-n:] will work fine when n is greater than one (e.g., somelist[-3:]). However, when n is zero, the expression so melist[-0:] will result in a copy of the original list.

Slicing can be extended to any Python class that implements the__getitem__ and __setitem__ special methods

In a file, functions and classes should be separated by two blank lines.

The basic principle is that you patch where an object is looked up, which is not necessarily the same place as where it is defined. A couple of examples will help to clarify this.

Given we put the system in a known state before the user (or external system) starts interacting with the system (in the When steps). Avoid talking about user interaction in givens. When we take key actions the user (or external system) performs. This is the interaction with your system which should (or perhaps should not) cause some state to change. Then we observe outcomes.

The current version is None. Mock is stable and widely used.

Up for a Challenge?

Python is in fact compiled to bytecode

A now-famous early usage of Python was in 1996: Google’s first successful web crawler.

Python was created by Guido Van Rossum in the early 90s. It is now one of the most popular languages in existence. I fell in love with Python for its syntactic clarity. It’s basically executable pseudocode.

The goal of this site is to provide a set of materials in support of my Python for Informatics: Exploring Information book to allow you to learn Python on your own. This page serves as an outline of the materials to support the textbook.

The hard part is teaching the consequences of each choice.

Read about how to use all of the things you installed.

A class instance has a namespace implemented as a dictionary which is the first place in which attribute references are searched. When an attribute is not found there, and the instance’s class has an attribute by that name, the search continues with the class attributes.

Compared with the 1.0 release, there were relatively few backwards incompatible changes, but there are still a few issues to be aware of with this major release.

The ability to seamlessly insert IPython notebooks into posts was one of the biggest drivers of my switch to Pelican.

Three things are required to fully guarantee a repeatable installation using requirements files.