863 Matching Annotations
  1. Last 7 days
    1. Python reports only one error message at a time—so the game is which error message will be reported first?

      Here is the buggy program:

      python 1 / 0 print() = None if False ñ = "hello

      Each line of code generates a different error message:

      • 1 / 0 will generate ZeroDivisionError: division by zero.
      • print() = None will generate SyntaxError: cannot assign to function call.
      • if False will generate SyntaxError: expected ':'.
      • ñ = "hello will generate SyntaxError: EOL while scanning string literal.

      The question is… which will be reported first?

      Spoilers: the specific version of Python matters (more than I thought it would) so keep that in mind if you see different results.

    1. The tokenizer takes your source code and chunks it into “tokens”. Tokens are just small pieces of source code that you can identify in isolation. As examples, there will be tokens for numbers, mathematical operators, variable names, and keywords (like if or for). The parser will take that linear sequence of tokens and essentially reshape them into a tree structure (that's what the T in AST stands for: Tree). This tree is what gives meaning to your tokens, providing a nice structure that is easier to reason about and work on. As soon as we have that tree structure, our compiler can go over the tree and figure out what bytecode instructions represent the code in the tree. For example, if part of the tree represents a function, we may need a bytecode for the return statement of that function. Finally, the interpreter takes those bytecode instructions and executes them, producing the results of our original program.
    2. Recap

      In this article you started implementing your own version of Python. To do so, you needed to create four main components:

      A tokenizer: * accepts strings as input (supposedly, source code); * chunks the input into atomic pieces called tokens; * produces tokens regardless of their sequence making sense or not.

      A parser: * accepts tokens as input; * consumes the tokens one at a time, while making sense they come in an order that makes sense; * produces a tree that represents the syntax of the original code.

      A compiler: * accepts a tree as input; * traverses the tree to produce bytecode operations.

      An interpreter: * accepts bytecode as input; * traverses the bytecode and performs the operation that each one represents; * uses a stack to help with the computations.

    3. Each bytecode is defined by two things: the type of bytecode operation we're dealing with (e.g., pushing things on the stack or doing an operation); and the data associated with that bytecode operation, which not all bytecode operations need.
    4. The interpreter accepts a list of bytecode operations and its method interpret will go through the list of bytecodes, interpreting one at a time.

      ``` from .compiler import Bytecode, BytecodeType


      class Interpreter: def init(self, bytecode: list[Bytecode]) -> None: self.stack = Stack() self.bytecode = bytecode self.ptr: int = 0

      def interpret(self) -> None:
          for bc in self.bytecode:
              # Interpret this bytecode operator.
              if bc.type == BytecodeType.PUSH:
              elif bc.type == BytecodeType.BINOP:
                  right = self.stack.pop()
                  left = self.stack.pop()
                  if bc.value == "+":
                      result = left + right
                  elif bc.value == "-":
                      result = left - right
                      raise RuntimeError(f"Unknown operator {bc.value}.")


    5. The interpreter is the part of the program that is responsible for taking bytecode operations as input and using those to actually run the source code you started off with.
    6. To write our compiler, we'll just create a class with a method compile. The method compile will mimic the method parse in its structure. However, the method parse produces tree nodes and the method compile will produce bytecode operations.
    7. The compiler is the part of our program that will take a tree (an AST, to be more precise) and it will produce a sequence of instructions that are simple and easy to follow.
    8. Instead of interpreting the tree directly, we'll use a compiler to create an intermediate layer.
    9. After we have our sequence of operations (bytecodes), we will “interpret” it. To interpret the bytecode means that we go over the bytecode, sequence by sequence, and at each point we perform the simple operation that the bytecode tells us to perform.
    10. Bytecodes are just simple, atomic instructions that do one thing, and one thing only.
    11. Abstract syntax tree

      It's an abstract syntax tree because it is a tree representation that doesn't care about the original syntax we used to write the operation. It only cares about the operations we are going to perform.

    12. The parser is the part of our program that accepts a stream of tokens and makes sure they make sense.
    13. The tokenizer

      The tokenizer is the part of your program that accepts the source code and produces a linear sequence of tokens – bits of source code that you identify as being relevant.

    14. The four parts of our program
      • Tokenizer takes source code as input and produces tokens;
      • Parser takes tokens as input and produces an AST;
      • Compiler takes an AST as input and produces bytecode;
      • Interpreter takes bytecode as input and produces program results.
    1. Once an interpreter is running (remembering what I said that it is preferable to leave them running) you can share data using a channel. The channels module is also part of PEP554 and available using a secret-import:

      ``` import _xxsubinterpreters as interpreters import _xxinterpchannels as channels

      interp_id = interpreters.create(site=site) channel_id = channels.create()

      interpreters.run_string( interp_id, """ import _xxinterpchannels as channels channels.send('hello!') """, shared={ "channel_id": channel_id } )

      print(channels.recv(channel_id)) ```

    2. To share data, you can use the shared argument and provide a dictionary with shareable (int, float, bool, bytes, str, None, tuple) values:

      ``` import _xxsubinterpreters as interpreters

      interp_id = interpreters.create(site=site)

      interpreters.run_string( interp_id, "print(message)", shared={ "message": "hello world!" } )

      interpreters.run_string( interp_id, """ for message in messages: print(message) """, shared={ "messages": ("hello world!", "this", "is", "me") } )

      interpreters.destroy(interp_id) ```

    3. To start an interpreter that sticks around, you can use interpreters.create() which returns the interpreter ID. This ID can be used for subsequent .run_string calls:

      ``` import _xxsubinterpreters as interpreters

      interp_id = interpreters.create(site=site)

      interpreters.run_string(interp_id, "print('hello world')") interpreters.run_string(interp_id, "print('hello universe')")

      interpreters.destroy(interp_id) ```

    4. Starting a sub interpreter is a blocking operation, so most of the time you want to start one inside a thread.

      ``` from threading import Thread import _xxsubinterpreters as interpreters

      t = Thread(target=interpreters.run, args=("print('hello world')",)) t.start() ```

    5. You can create, run and stop a sub interpreter with the .run() function which takes a string or a simple function

      ``` import _xxsubinterpreters as interpreters

      interpreters.run(''' print("Hello World") ''') ```

    6. Inter-Worker communication

      Whether using sub interpreters or multiprocessing you cannot simply send existing Python objects to worker processes.

      Multiprocessing uses pickle by default. When you start a process or use a process pool, you can use pipes, queues and shared memory as mechanisms to sending data to/from the workers and the main process. These mechanisms revolve around pickling. Pickling is the builtin serialization library for Python that can convert most Python objects into a byte string and back into a Python object.

      Pickle is very flexible. You can serialize a lot of different types of Python objects (but not all) and Python objects can even define a method for how they can be serialized. It also handles nested objects and properties. However, with that flexibility comes a performance hit. Pickle is slow. So if you have a worker model that relies upon continuous inter-worker communication of complex pickled data you’ll likely see a bottleneck.

      Sub interpreters can accept pickled data. They also have a second mechanism called shared data. Shared data is a high-speed shared memory space that interpreters can write to and share data with other interpreters. It supports only immutable types, those are:

      • Strings
      • Byte Strings
      • Integers and Floats
      • Boolean and None
      • Tuples (and tuples of tuples)

      To share data with an interpreter, you can either set it as initialization data or you can send it through a channel.

    7. The next point when using a parallel execution model like multiprocessing or sub interpreters is how you share data.

      Once you get over the hurdle of starting one, this quickly becomes the most important point. You have two questions to answer:

      • How do we communicate between workers?
      • How do we manage the state of workers?
    8. Half of the time taken to start an interpreter is taken up running “site import”. This is a special module called site.py that lives within the Python installation. Interpreters have their own caches, their own builtins, they are effectively mini-Python processes. Starting a thread or a coroutine is so fast because it doesn’t have to do any of that work (it shares that state with the owning interpreter), but it’s bound by the lock and isn’t parallel.
    9. Both multiprocessing processes and interpreters have their own import state. This is drastically different to threads and coroutines. When you await an async function, you don’t need to worry about whether that coroutine has imported the required modules. The same applies for threads.

      For example, you can import something in your module and reference it from inside the thread function:

      ```python import threading from super.duper.module import cool_function

      def worker(info): # This already exists in the interpreter state cool_function()

      info = {'a': 1} thread = Thread(target=worker, args=(info, )) ```

    10. Another important point is that multiprocessing is often used in a model where the processes are long-running and handed lots of tasks instead of being spawned and destroyed for a single workload. One great example is Gunicorn, the popular Python web server. Gunicorn will spawn “workers” using multiprocessing and those workers will live for the lifetime of the main process. The time to start a process or a sub interpreter then becomes irrelevant (at 89 ms or 1 second) when the web worker can be running for weeks, months or years. The ideal way to use these parallel workers for small tasks (like handle a single web request) is to keep them running and use a main process to coordinate and distribute the workload
    11. What is the difference between threading, multiprocessing, and sub interpreters?

      The Python standard library has a few options for concurrent programming, depending on some factors:

      • Is the task you’re completing IO-bound (e.g. reading from a network, writing to disk)
      • Does the task require CPU-heavy work, e.g. computation
      • Can the tasks be broken into small chunks or are they large pieces of work?

      Here are the models:

      • Threads are fast to create, you can share any Python objects between them and have a small overhead. Their drawback is that Python threads are bound to the GIL of the process, so if the workload is CPU-intensive then you won’t see any performance gains. Threading is very useful for background, polling tasks like a function that waits and listens for a message on a queue.
      • Coroutines are extremely fast to create, you can share any Python objects between them and have a miniscule overhead. Coroutines are ideal for IO-based activity that has an underlying API that supports async/await.
      • Multiprocessing is a Python wrapper that creates Python processes and links them together. These processes are slow to start, so the workload that you give them needs to be large enough to see the benefit of parallelising the workload. However, they are truly parallel since each one has it’s own GIL.
      • Sub interpreters have the parallelism of multiprocessing, but with a much faster startup time.
  2. Nov 2023
    1. Rosetta is now Generally Available for all users on macOS 13 or later. It provides faster emulation of Intel-based images on Apple Silicon. To use Rosetta, see Settings. Rosetta is enabled by default on macOS 14.1 and later.

      Tested it on my side, and poetry install of one Python project took 44 seconds instead of 2 minutes 53 seconds, so it's nearly a 4x speed increase!

  3. Oct 2023
    1. Method 1: numpy.any() to check if the NumPy array is empty in Python numpy.any() method is used to test whether any array element along a given axis evaluates to True. Syntax: numpy.any(a, axis = None, out = None, keepdims = <no value>) Parameters: array: Input array whose elements need to be checked.axis: Axis along which array elements are evaluated.out: Output array having the same dimensions as Input arraykeepdmis: If this is set to True, the axes which are reduced are left in the result. Return Value: A new Boolean array (depending on the ‘out;’ parameter) 1234567import numpy as nparr = np.array([])flag = not np.any(arr)if flag:    print('Array is empty')else:    print('Array is not empty') Output: Array is empty In this example, we have used numpy.any() method to check whether the array is empty or not. As the array is empty, the value of the flag variable becomes True, and so the output ‘Array is empty’ is displayed. The limitation to this function is that it does not work if the array contains the value 0 in it.

      This is WRONG.

      numpy.any() checks if there is at least one non-zero element in an array.

    1. nserting a new chicken into a ring at some specified location in it, usu-ally first or last.2. Removing a chicken from a ring.3. Putting all the chickens of one ring, in order, into another at some speci-fied location in it, usually first or last.4. Performing some auxiliary operation on each member of a ring in eitherforward or reverse order

      In simpler terms, Sutherland's thesis is discussing the basic operations of a data structure known as a ring. A ring is a type of list where the elements are connected in a circular manner. The operations he mentions are:

      1. Inserting a new element into the ring at a specified location.
      2. Removing an element from the ring.
      3. Moving all elements of one ring, in order, into another ring at a specified location.
      4. Performing an operation on each member of a ring in either forward or reverse order.

      These operations are implemented using macro instructions in the compiler language. A macro instruction is a directive to the compiler which specifies how an input pattern should be mapped to an output pattern.

      The thesis also discusses the generation of new elements. Subroutines are used to set up new elements in free spaces in the storage structure. When parts of the drawing are deleted, the registers representing them become free and are placed in a 'FREES' ring. New components are set up at the end of the storage area, while free blocks are allowed to accumulate. A process called 'garbage collection' periodically compacts the storage structure by removing the free blocks and relocating the information above them.

      In Python, the ring data structure can be implemented using a doubly linked list. Here's a simple example:

      ```python class Node: def init(self, data): self.data = data self.next = None self.prev = None

      class Ring: def init(self): self.head = None

      def append(self, data):
          if not self.head:
              self.head = Node(data)
              self.head.next = self.head
              self.head.prev = self.head
              new_node = Node(data)
              new_node.prev = self.head.prev
              new_node.next = self.head
              self.head.prev.next = new_node
              self.head.prev = new_node
      def display(self):
          temp = self.head
          while True:
              print(temp.data, end = " ")
              temp = temp.next
              if temp == self.head:


      In this example, the Node class represents an element in the ring, and the Ring class represents the ring itself. The append method is used to add a new element to the ring, and the display method is used to print all elements in the ring.


      Ivan writes about a concept called "Ring Structure". This is a way of organizing and linking different elements or components in a system.

      In simpler terms, imagine you have a bunch of different objects (like points and lines in a drawing program like Sketchpad). You want to keep track of how these objects are related to each other. For example, you might want to know all the lines that end at a particular point.

      To do this, Ivan uses a "ring structure". Each object has a "string of pointers" - basically a list of references to other objects. This list is circular - the last item in the list points back to the first item. This makes it easy to move forwards and backwards through the list.

      Each object has two "registers" or slots for keeping track of these relationships. One slot is for the object itself, and the other is for the list of related objects.

      Ivan uses the terms "hen" and "chicken" to describe these slots. The "hen" is the object itself, and the "chicken" is the list of related objects.

      Here's a simple Python code example to illustrate this concept:

      ```python class Point: def init(self): self.hen = self self.chickens = []

      class Line: def init(self, point1, point2): self.hen = self self.chickens = [point1, point2] point1.chickens.append(self) point2.chickens.append(self) ```

      In this example, a Point object has a hen that refers to itself and a list of chickens that will contain any Line objects that end at this point. When a Line is created, it adds itself to the chickens list of its end points.

      The "ring structure" is a way to organize and link different elements in a system, making it easier to find and update related elements.


      Mnemonics for Registers: Instead of remembering numerical indices, we use human-readable keys.

      python point = {'TYPE': 'Point', 'PVAL_X': 5, 'PVAL_Y': 10}

      Flexibility: If we want to change the internal structure, we can easily do so by changing the keys.


      Changing 'PVAL_X' to 'X_COORD'

      point = {'TYPE': 'Point', 'X_COORD': 5, 'PVAL_Y': 10} ```


      • The first component ('TYPE') indicates the type of element.
      • Numerical information like coordinates is stored at the end.

      python line = {'TYPE': 'Line', 'START': 'point1', 'END': 'point2', 'LENGTH': 7.2}

      Pointers and Topology: We can use pointers (references) to other elements to establish relationships.

      python point1 = {'TYPE': 'Point', 'PVAL_X': 1, 'PVAL_Y': 1} point2 = {'TYPE': 'Point', 'PVAL_X': 4, 'PVAL_Y': 5} line = {'TYPE': 'Line', 'START': point1, 'END': point2}

      Relocation: If we move point1 to a new variable, we update the pointer in line.

      python new_point1 = point1 # Relocating point1 to new_point1 line['START'] = new_point1 # Updating the pointer

      Segregation of Data: Numerical data is at the end, so if we need to move elements, the numerical data remains untouched.


      Even if we relocate, the numerical data ('PVAL_X' and 'PVAL_Y') remains the same.


  4. Sep 2023
    1. Configuring PyCharm: Open PyCharm with ‘Pytest Web Framework’ Press Ctrl+Alt+S > Project Click ‘Project Interpreter’ Select Python 3.6 Click ‘OK’ Go to write over 100500 automated tests!!!

      This section provides a step-by-step guide on setting up PyCharm for automated testing using the 'Pytest Web Framework'.

  5. Aug 2023
  6. Jul 2023
    1. cat requirements.txt | grep -E '^[^# ]' | cut -d= -f1 | xargs -n 1 poetry add

      Use poetry init to create a sample pyproject.toml, and then trigger this line to export requirements.txt into a pyproject.toml

    1. As you can see, it has sliced along axis 0, the first axis. A slice, therefore, selects a range of elements along an axis. It can be helpful to read the expression arr2d[:2] as "select the first two rows of arr2d."

      Slices follow a similar logic than indexing in NumPy array's. array[:2] selects a range of elements along a single axis,, but array[:2, 1:] does it along two axis.

    1. You might want to suppress only ValueError, since a TypeError (the input was not a string or numeric value) might indicate a legitimate bug in your program. To do that, write the exception type after except: def attempt_float(x): try: return float(x) except ValueError: return x
    2. Since generators produce output one element at a time versus an entire list all at once, it can help your program use less memory.
    3. It is not until you request elements from the generator that it begins executing its code:

      A generator is a function-like iterator object.

    4. A generator is a convenient way, similar to writing a normal function, to construct a new iterable object. Whereas normal functions execute and return a single result at a time, generators can return a sequence of multiple values by pausing and resuming execution each time the generator is used. To create a generator, use the yield keyword instead of return in a function:
    5. In this case, return_value would be a 3-tuple with the three returned variables. A potentially attractive alternative to returning multiple values like before might be to return a dictionary instead:

      Returning multiple values in Python is expressed as a tuple by default and each value is correspondingly assigned. Optionally, you can return a dictionary if specified.

    6. The for parts of the list comprehension are arranged according to the order of nesting, and any filter condition is put at the end as before.

      Nested list comprehensions follow the same logic as nested for loops. The difference strives that in list comprehensions the filtered variable is mentioned twice.

    1. python -m calendar

      So surprised that you can output a calendar view using Python

    2. python -m site, which outputs useful information about your installation

      python -m site <--- see useful information about your Python installation

    1. ```python from flask import Flask, request from collections import defaultdict import re import random

      GREEN ="🟩" YELLOW ="🟨" WHITE ="⬜"

      def get_answers(): with open("allowed_answers.txt") as f: answers = set(l for l in f.read().splitlines() if l) return answers

      def get_guesses(): guesses = get_answers() with open("allowed_guesses.txt") as f: for l in f.read().splitlines(): if l: guesses.add(l) return guesses

      app = Flask(name, static_folder="static") app.answers = get_answers() app.guesses = get_guesses() word = random.choice(list(app.answers)) print(f"The word is {word}")

      def with_header(content): return f"""

      <html> <head> <link rel="search" type="application/opensearchdescription+xml" title="searchGame" href="http://searchgame:5000/static/opensearch.xml" /> </head> <body> {content} </body></html>


      @app.route("/") def home(): return with_header("

      Right click on the address bar to install the search engine.


      @app.route("/search") def search(): return with_header(f"Content: {request.args.get('q')}")

      def to_result(guess, answer): chars = [WHITE] * 5 count = defaultdict(int) for idx, (g, a) in enumerate(zip(guess, answer)): if g == a: chars[idx] = GREEN else: count[a] += 1

      for idx, g in enumerate(guess):
          if g in count and count[g] > 0 and chars[idx] == WHITE:
              chars[idx] = YELLOW
              count[g] -= 1
      return "".join(chars)

      def maybe_error(guess): if len(guess) < 5: return f"less than 5 characters" if len(guess) > 5: return f"greater than 5 characters" if guess not in app.guesses: return f"not in wordlist" return None

      @app.route("/game") def game(): query = request.args.get("q") guesses = [x for x in re.split("[. ]", query) if x] response = [] if not guesses: response.append("Enter 5-letter guesses separated by spaces") else: most_recent = guesses[-1] # Don't show "too short" error for most recent guess if len(most_recent) < 5: guesses = guesses[:-1] if not guesses: response.append("Enter a wordle guess") for guess in guesses[::-1]: error = maybe_error(guess) if error is None: result = to_result(guess, word) s = f"{guess} | {result}" if result == GREEN * 5: s = f"{s} | CORRECT!" response.append(s) else: response.append(f"{guess} | ERROR: {error}")

      return [query, response]


    1. o con

      حالا می خواد بره با Axios وصلش کنه

    2. e

      حالا رفت سراغ Vue

    3. requests

      در واقع تعیین می کنه که درخواست ها از چه پروتکلی، با چه IP یا Domain Name و رو چه Port می تونی جواب بدی. خیلی خووبه براش تعیین کنی تو فقط باید از این IP و Port که در Front تعیین کردی استفاده کنی

    4. Flask-CORS

      تازه رفت سراغ Flask-origin چقد جاالب.

    5. pyth

      اول یه Virtual Environment می سازه

    1. For a new project, I’d just immediately start with Ruff; for existing projects, I would strongly recommend trying it as soon as you start getting annoyed about how long linting is taking in CI (or even worse, on your computer).

      Recommendation for when to use Ruff over PyLint or Flake8

  7. Jun 2023
    1. ```python def split_user(userid): """ Return the user and domain parts from the given user id as a dict.

      For example if userid is u'acct:seanh@hypothes.is' then return
      {'username': u'seanh', 'domain': u'hypothes.is'}'
      :raises InvalidUserId: if the given userid isn't a valid userid
      match = re.match(r"^acct:([^@]+)@(.*)$", userid)
      if match:
          return {"username": match.groups()[0], "domain": match.groups()[1]}
      raise InvalidUserId(userid)


    1. ```html

      <body> <div> {% for chat in chats %} <div>{{ chat.contents }}</div> <button id={{chat.id}} ✅ onClick=getId(id) ✅ > print this chat_id out </button> {% endfor %} </div> ... <script> function getId(id){ console.log(id) } </script> </body>


    1. Examples of frontends include: pip, build, poetry, hatch, pdm, flit Examples of backends include: setuptools (>=61), poetry-core, hatchling, pdm-backend, flit-core

      Frontend and backend examples of Python's build backends

    2. pyproject.toml-based builds are the future, and they promote better practices for reliable package builds and installs. You should prefer to use them!

      setup.py is considered a "legacy" functionality these days

    3. Did you say setuptools? Yes! You may be familiar with setuptools as the thing that used your setup.py files to build packages. Setuptools now also fully supports pyproject.toml-based builds since version 61.0.0. You can do everything in pyproject.toml and no longer need setup.py or setup.cfg.

      setuptools can now utilize pyproject.toml

  8. May 2023
    1. With this dataclass, I have an explicit description of what the function returns.

      Dataclasses give you a lot more clarity of what the function returns, in comparison to returning tuples or dictionaries

    1. Host machine: docker run -it -p 8888:8888 image:version Inside the Container : jupyter notebook --ip --no-browser --allow-root Host machine access this url : localhost:8888/tree‌

      3 ways of running jupyter notebook in a container

    1. How can I add, subtract, and compare binary numbers in Python without converting to decimal?

      I think the requirements of this were not spelled out well. After reading this over a couple of times, I think the problem should be…

      "Add, subtract, and compare binary numbers in Python as strings, without converting them to decimal."

      I'll take on that problem sometime when I get free time!

    1. 'handlers': { 'console': { 'level': 'INFO', 'class': 'logging.StreamHandler', 'stream': sys.stdout, 'formatter': 'verbose' }, },

      It's as simple as adding "sys.stdout" to the "stream" attribute.

  9. Apr 2023
    1. If you install a package with pip’s --user option, all its files will be installed in the .local directory of the current user’s home directory.

      One of the recommendations for Python multi-stage Docker builds. Thanks to pip install --user, the packages won't be spread across 3 different paths.

  10. Mar 2023
    1. Honestly, all the activation scripts do are:

      See the 4 steps below to understand what activating an environment in Python really does

    1. Using pex in combination with S3 for storing the pex files, we built a system where the fast path avoids the overhead of building and launching Docker images.Our system works like this: when you commit code to GitHub, the GitHub action either does a full build or a fast build depending on if your dependencies have changed since the previous deploy. We keep track of the set of dependencies specified in setup.py and requirements.txt.For a full build, we build your project dependencies into a deps.pex file and your code into a source.pex file. Both are uploaded to Dagster cloud. For a fast build we only build and upload the source.pex file.In Dagster Cloud, we may reuse an existing container or provision a new container as the code server. We download the deps.pex and source.pex files onto this code server and use them to run your code in an isolated environment.

      Fast vs full deployments

    1. Well, in short, with iterators, the flow of information is one-way only. When you have an iterator, all you can really do call the __next__ method to get the very next value to be yielded. In contrast, the flow of information with generators is bidirectional: you can send information back into the generator via the send method.
      • Iterator ← one-way communication (can only yield stuff)
      • Generator ← bidirectional communication (can also accept information via the send method)
    1. PythonCopyconfigs = {"fs.azure.account.auth.type": "OAuth", "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider", "fs.azure.account.oauth2.client.id": "<application-id>", "fs.azure.account.oauth2.client.secret": dbutils.secrets.get(scope="<scope-name>",key="<service-credential-key-name>"), "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/<directory-id>/oauth2/token"} # Optionally, you can add <directory-name> to the source URI of your mount point. dbutils.fs.mount( source = "abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/", mount_point = "/mnt/<mount-name>", extra_configs = configs)
    1. So why aren't more people using Nim? I don't know! It's the closest thing to a perfect language that I've used by far.

      Nim sounds as the most ideal language when comparing to Python, Rust, Julia, C#, Swift, C

    1. depending on how smart the framework is, you might find yourself installing Conda packages over and over again on every run. This is inefficient, even when using a faster installer like Mamba.
    2. there’s the bootstrapping problem: depending on the framework you’re using, you might need to install Conda and the framework driver before you can get anything going. A Docker image would come prepackaged with both, in addition to your code and its dependencies. So even if your framework supports Conda directly, you might want to use Docker anyway.
    3. The only thing that will depend on the host operating system is glibc, pretty much everything else will be packaged by Conda. So a pinned environment.yml or conda-lock.yml file is a reasonable alternative to a Docker image as far as having consistent dependencies.

      Conda can be a sufficient alternative to Docker

    4. To summarize, for the use case of Python development environments, here’s how you might choose alternatives to Docker:

      (see table below)

    5. Conda packages everything but the standard C library, from C libraries to the Python interpreter to command-line tools to compilers.
    1. When you call 'foo' in Ruby, what you're actually doing is sending a message to its owner: "please call your method 'foo'". You just can't get a direct hold on functions in Ruby in the way you can in Python; they're slippery and elusive. You can only see them as though shadows on a cave wall; you can only reference them through strings/symbols that happen to be their name. Try and think of every method call 'object.foo(args)' you do in Ruby as the equivalent of this in Python: 'object.getattribute('foo')(args)'.
    2. def document(f): def wrap(x): print "I am going to square", x f(x) return wrap @document def square(x): print math.pow(x, 2) square(5)
    1. Post involved with finding a good jupyter notebook visualization lib for DAGs.

  11. Feb 2023
    1. What object-oriented means

      What does the object-oriented means? Objects are models of somethings that can do certain things and have certain things done to them. Formally, an object is a collection of data and associated behaviors.

    2. The difference between object-oriented design and object-orientedprogramming

      What is the design and programming mean in OOP?



  12. Jan 2023
    1. The len() function returns the length of a string:

      This function may be useful if it is necessary to determine if a string is too long or too short for a given task.

    1. Float is a function or reusable code in the Python programming language that converts values into floating point numbers. Floating point numbers are decimal values or fractional numbers like 133.5, 2897.11, and 3571.213, whereas real numbers like 56, 2, and 33 are called integers.

      Meaning of Float in Python proramming language

    1. make dev

      I am getting this error while doing make dev I have installed the required version of python, still update with the required version please.

      File "/home/ec2-user/projects/h/h/search/config.py", line 213, in _ensure_icu_plugin

      names = [x.strip() for x in conn.cat.plugins(h="component").split("\n")]

      AttributeError: 'list' object has no attribute 'split'

    1. Now, if you try to parse a date (using the pandas.to_datetime() function) that lies outside this range, we get the above ParseError.

      The datetime type in pandas can only take values inside a given range, for example, dates less than 1677-09-21 and greater than 2262-04-11 cannot be used in Pandas. Is this due to the bit size the datetime[ns] type uses in Pandas?

    1. Python dictionaries are another collection. Real word dictionaries are a good analogy to understand them: they contain a list of items, each item has a key and a value. In the traditional dictionary, the key is the word and the value is the explanation or description of it. In Python you can do something similar.

      Meaning of Dictionary according to Python Language

    1. CamelCase is formally referred to as medial capitals. It may also be called or styled as PascalCase, camel case, InterCaps, mixedCase or WikiCase.

      this is a descripton

    2. CamelCase is a way to separate the words in a phrase by making the first letter of each word capitalized and not using spaces

      Camel Case is a method to avoid spaces in names. For example UpperCamelCase and lowerCamelCase

      All PascalCase styles are CamelCase but not all CamelCase are PascalCase

    1. Note: Triple quotes, according to official Python documentation are docstrings, or multi-line docstrings and are not considered comments. Anything inside triple quotes is read by the interpreter. When the interpreter encounters the hash symbol, it ignores everything after that. That is what a comment is defined to be. 
    1. Snake case (stylized as snake_case) refers to the style of writing in which each space is replaced by an underscore (_) character, and the first letter of each word written in lowercase. It is a commonly used naming convention in computing, for example for variable and subroutine names, and for filenames.
    1. PascalCase

      I typically like to use PascalCase to indicate variables in my code. It is common in large development environments to have a coding standard document that defines how each programmer on the team will use various conventions

    1. Python — Json documents double as valid Python literals with minimal adaptation, and Python supports trailing commas and comments. It has variables and functions, powerful string interpolation, and json.dump built in. A self-contained Python file that prints json to stdout goes a long way!

      If you would like to template yaml, then rather do it in Nix or Python

    1. No, in Python "[ [] foo [] boo [][]] ".strip(" []") returns "foo [] boo".

      I would have expected it would remove the string " []", not the occurrences of any of the characters within the string...

    2. There is no such method in ruby, but you can easily define it like: def my_strip(string, chars) chars = Regexp.escape(chars) string.gsub(/\A[#{chars}]+|[#{chars}]+\z/, "") end
    1. Solution #3: Switch to Conda-Forge

      Yet another possible solution for M1 users, but you need to use conda and expect less packages than in PyPI

    2. In general, the environment variable is too heavy-handed and should be avoided, since it will impact all images you build or run. Given the speed impact, you don’t for example want to run your postgres image with emulation, to no benefit.

      Which options to avoid

    3. The problem with this approach is that the emulation slows down your runtime. How much slower it is? Once benchmark I ran was only 6% of the speed of the host machine!

      Speed is the core problem with emulation

    4. The other option is to run x86_64 Docker images on your ARM64 Mac machine, using emulation. Docker is packaged with software that will translate or emulate x86_64 machine code into ARM64 machine code on the fly; it’s slow, but the code will run.

      Another possible solution for M1 users (see snippets below)

    5. Third, you can pre-compile wheels, store them somewhere, and install those directly instead of downloading the packages from PyPI.

      Third possible solution for M1 users

    6. If you have a compiler installed in your Docker image and any required native libraries and development headers, you can compile a native package from the source code. Basically, you add a RUN apt-get upgrade && apt-get install -y gcc and iterate until the package compiles successfully.

      Second possible solution for M1 users

    7. First, it’s possible the relevant wheels are available in newer versions of the libraries.

      First possible solution for M1 users

    8. When you pip install murmurhash==1.0.6 on a M1/M2 Mac inside Docker, again it looks at the available files

      Other possible steps that pip will do when trying to install a Python package without a relevant CPU instruction set

    9. When you pip install filprofiler==2022.05.0 on a M1/M2 Mac inside Docker, pip will look at the available files for that version, and then

      3 steps that pip will do when trying to install a Python package without a relevant CPU instruction set

    10. In either case, pure Python will Just Work, because it’s interpreted at runtime: there’s no CPU-specific machine code, it’s just text that the Python interpreter knows how to run. The problems start when we start using compiled Python extensions. These are machine code, and therefore you need a version that is specific to your particular CPU instruction set.

      M1 Python issues

    1. Data Viz with Python and RLearn to Make Plots in Python and R

      data viz with python and R

    1. Here's my opinion, having written many thousands of lines of mypy code.

      Negative opinion on mypy (see below this annotation)

  13. Dec 2022
    1. Overall, the code was significantly shorter compared to the tkinter version I did last year. That version had a few more features, but I'd say Textual felt much easier to reason about.

      Textual is much easier than tkinter.

    1. Or more directly

      Hack for pasting multiline Python scripts in a terminal:

      1. python
      2. exec('''<paste code>''')
      3. [ENTER]
    1. In Python, everything is an object – integers, strings, lists, functions, even classes themselves.
    1. Try this code at app startup:

      Code to improve Python GC settings to increase the performance by 20%

    2. The trigger is when you allocate 700 or more container objects (classes, dicts, tuples, lists, etc) more than have been cleaned up, a GC cycle runs.

      Trigger for GC runs in Python

    1. To summarize the three options we’ve seen, as well as a streaming ijson-based solution:

      Comparison of 4 Python's JSON libraries

    1. Always start with functionsGrow to classes once you feel you can group different subsets of functions

      Python rules for creating a function or a class

    2. First of all, in Python there are no such things as "files" and I noticed this is the main source of confusion for beginners.If you're inside a directory that contains any __init__.py it's a directory composed of modules, not files.

      On "files" in Python

    1. the fact that the Poetry developers intentionally introduced failures to people’s CI/CD pipelines to motivate them to move away from Poetry’s legacy installer… Though we didn’t rely on the installer in our pipelines, this was the death knell for the continued use of Poetry.

      Video on this topic: https://youtu.be/Gr9o8MW_pb0