Episódios

  • Watch the live stream:

    Watch on YouTube

    About the show

    Sponsored by us! Support our work through:

    Our courses at Talk Python Trainingpytest bookPatreon Supporters

    Special guest

    Guy Royse

    Brian #1: How to make an awesome Python package in 2021

    Anton Zhiyanov, @ohmypyAlso thanks John Mitchell, @JohnTellsAll for posting about it.Great writing taking you through everything in a sane order.Stubbing a projectwith just .gitignore and a directory with a stub __init__.py.Test packaging and publishinguse flit init to create initial pyproject.tomlset up your ~/.pypirc filepublish to the test repoMake the real thing make an implementationpublishExtrasAdding README.md & CHANGELOG.md and updating pyproject.toml to include README.md and a Python version selector.Adding linting and testing with pytest, tox, coverage, and othersBuilding in the cloud with GH Actions, Codecov, Code ClimateAdding badgesTask automation with a MakefilePublishing to PyPI from a GH ActionMissing (but possibly obvious):GitHub projectChecking your project name on PyPI earlySuper grateful for:Do all of this early in the projectUsing flit publish --repository pypitest and spelling out how to set up a ~/.pypirc file.Start to finish workflowExample project with all filled out project files

    Michael #2: KubestrikerKubestriker performs numerous in depth checks on kubernetes infra to identify the security misconfigurations

    Focuses on running in production and at scale.kubestriker is Platform agnostic and works equally well across more than one platform such as self hosted kubernetes, Amazon EKS, Azure AKS, Google GKE etc.Current CapabilitiesScans Self Managed and cloud provider managed kubernetes infraReconnaissance phase checks for various services or open portsPerforms automated scans incase of insecure, readwrite or readonly services are enabledPerforms both authenticated scans and unauthenticated scansScans for wide range of IAM Misconfigurations in the clusterScans for wide range of Misconfigured containersScans for wide range of Misconfigured Pod Security PoliciesScans for wide range of Misconfigured Network policiesScans the privileges of a subject in the clusterRun commands on the containers and streams back the outputProvides the endpoints of the misconfigured servicesProvides possible privilege escalation detailsElaborative report with detailed explanation

    Guy #3: wasmtime

    WebAssembly runtime with support for:Python, Rust, C, Go, .NETDocumentation here: https://docs.wasmtime.dev/Supports WASI (Web Assembly System Interface):WASI supports IO operations—it does for WebAssembly what Node.js did for JavaScript

    Brian #4: Depend-a-lot-bot

    Anthony Shaw, @anthonypjshawA bot for GitHub that automatically approves + merges PRs from dependabot and PyUp.io when they meet certain criteria:All the checks are passingThe package is on a safe-list (see configuration)Example picture shows an auto approval and merge of a tox version update, showing “This PR looks good to merge automatically because tox is on the save-list for this repository”.Configuration in a .yml file. I learned recently that most programming jobs that can be automated eventually devolve into configuring a yml file.

    Michael #5: Supreme Court sides with Google in API copyright battle with Oracle

    The Supreme Court has sided with Google in its decade-long legal battle with Oracle over the copyright status of application programming interfaces. The ruling means that Google will not owe Oracle billions of dollars in damages. It also has big implications for the broader software industryThe ruling heads off an expected wave of lawsuits over API copyrights.The case dates back to the creation of the Android platform in the mid-2000s.Google independently implemented the Java API methods, but to ensure compatibility, it copied Java's method names, argument types, and the class and package hierarchy.Over a decade of litigation, Google won twice at the trial court level, but each time, the ruling was overruled by the Federal Circuit appeals court. The case finally reached the Supreme Court last year.Writing for a six-justice majority, Justice Stephen Breyer held that Google's copying of the Java API calls was permissible under copyright's fair use doctrine.

    Guy #6: RedisAI

    Module for Redis that add AI capabilitiesTurns Redis into a model server:Supports TF, PyTorch, and ONNX modelsAdds the TENSOR data typeONNX + Redis has positive architectural implications

    Extras

    Michael

    git for WindowsJupyterLab reaches v3 (via via Allan Hansen)Why not support Python letter by Brian SkinnDjango 3.2 is out & is LTSPyCharm 2021.1 just dropped with Code With Me

    Brian

    The PSF is hiring a Developer-in-Residence to support CPython!

    Joke

    Vim Escape RoomsHappiness-
  • Watch the live stream:

    Watch on YouTube

    About the show

    Sponsored by us!Special guest: Micaela Reyes

    Brian #1: Number One, that's "retract plank," not "remove plank."

    Yanking vs removing versions on PyPIhttps://twitter.com/nedbat/status/1376901333958201352?s=20https://pypi.org/help/#yankedsee also https://doughellmann.com/posts/so-youve-released-a-broken-package-to-pypi-what-do-you-do-now/

    Michael #2: SQLAlchemy 1.4.0 Released

    Exciting: 1st release to properly support an async APIHas a new select() + execute() rather than session.query() APIIntended to unify Core and ORM.See new vs. old API compared.Requires aiosqlite for async API + SQLite: conn_str = 'sqlite+aiosqlite:///filename'

    Micaela #3: django-tenants

    by Tom TurnerMulti-tenancy Implementation for Django (typically for SaaS websites e.g. Shopify)currently on v3.2.1 (Aug 2020) releaseRequirements: Django 2 and PostgreSQLIt was largely based on django-tenant-schemas libraryData Architecture: shared database, separate schema for each tenantDomain setup / URL routing for root and per tenant:Examples:http://my-domain.com:8000/http://tenant.my-domain.com:8000/http://tenant2.my-domain.com:8000/Possible Use-cases:a hospital with different branchesrestaurant franchise with different branchesPossible Limitations:You can’t use the normal migration commands (python manage.py migrate_schemas instead of python manage.py migrate)Reports - when you need to create a report regarding all the clients/tenants, tenant data will be on separate schemassee also: How to get Django Tenants up and Running

    Brian #5: pre-commit ci

    pre-commit is “a framework for managing and maintaining multi-language pre-commit hooks.”Hooks can be run during commits, but also hooked other events, such as merge, push, after switching branches, etc.Even if you run pre-commit yourself, it’d be nice to make sure all hooks are run by people submitting pull requests.pre-commit ci *is a service that “enforces that these issues are discovered (which is opt-in for each developer's workflow via *pre-commit) but also fixes the issues automatically, letting developers focus their time on more valuable problems.”

    Michael #4: Snyk (Python) Package Advisor

    via David SmitProvides package health score, built up fromPopularityMaintenanceSecurityCommunityCode of conduct and related metricsNumber of releases, commits, maintainers, age, etc.Lots of nice graphs and ways you can contribute

    Micaela #6: PyWebIO

    by Wang WeiminAllows you build simple web applications or browser-based GUI applications without the need to write HTML and JavaScript.Has input and output modulesBased on tornadoPossible Use-cases:Hobby projects - for people who know how to write Python code but want to do away with JavaScriptCould be a good teaching tool - ex. if you’re teaching kids or beginners and you want to make them appreciate the concept of input and outputPossible Limitation: once you need to style your HTML, it might be a bit difficult to do so since it’s tightly coupled with the Python code

    Extras

    Michael

    absolufy-imports by Marco GorelliAnthony Shaw actually made a Beanie Baby in the terminal within a day of the last episode!PHP supply chain hack

    Brian

    packages: pytest-yuk, pytest-srcpaths, pytest-check

    Micaela

    Last day of March membership drive for the PSF https://twitter.com/ThePSF/status/1377000184375296003

    Joke: Commenting your code

  • Estão a faltar episódios?

    Clique aqui para atualizar o feed.

  • Special guests:

    Kelly Schuster-ParedesSean TiborWatch on YouTube

    Brian #1: DataClass vs NamedTuple vs Object: A Battle of Performance in Python

    Jack SongI’ve been using dataclass for a while now and love it. For some reason, I thought I heard there was some performance hit from them, so I was a bit worried before reading this.Jack came up with “a performance tests to compare the different size and speed when creating, reading and executing functions for Object, NamedTuple and the new DataClass introduced in Python 3.7” Object NamedTuple DataClass create 2.94 µs 2.01 µs 2.34 µs read property 24.7 ns 26.9 ns 24.7 ns nested property 48.1 ns 75.8 ns 52.1 ns execute function 829 ns 946 ns 821 ns size 56 bytes 80 bytes 56 bytesMarvelous. Dataclass is still awesome. At the very least, it’s on the same order of size and speed as other structures.Further questions:This was a limited bit of code, and performance metrics always depend on what kind of example code was used. If anyone has info about different performance examples that give wildly different results or even similar results, regarding DataClass, I’d love to hear about it.

    Michael #2: Can My Water Cooled Raspberry Pi Cluster Beat My MacBook?

    Did you know there were Raspberry Pi clusters? Amazing.Two primary things of interestPi clusters exist!All the tools and setup to make the pi cluster nodes run in a grid computing scenarioCompared against a mediocre HP laptop:10,000 → 1.69 sec100,000 → 74 sec200,000 → 268 secCompared against an Intel MB Air:10,000 → .88 sec100,000 → 83 sec200,000 → 355 secPi cluster, one node:10,000 → 1.57 sec100,000 → 148 sec200,000 → 646 secPi cluster, grid computed:10,000 → .65 sec100,000 → 22 sec200,000 → 85 secRan the same script on my Mac Mini M1 (on Python 3.9.2 Rosetta, single threaded)10,000 → .33 sec100,000 → 24 sec200,000 → 91 sec

    Ran the my parallel script on my Mac Mini M1 (on Python 3.9.2 Rosetta, multi-threaded)

    10,000 → .28 sec100,000 → 7 sec200,000 → 26 sec500,000 → 2m (vs. 9m on the cluster)

    Ran the my parallel script on my Alienware sim racing machine (16 core, multi-threaded)

    10,000 → .27 sec100,000 → 2.7 sec200,000 → 10 sec500,000 → 1mVideo of the Alienware sound

    Kelly #3: There is an app for that!

    I am always trying to find time to practice Python and learn more ways to teach Python Basics. I amalways on the go, so having Python Practice online and an online editor is always a plus.

    Programiz a company out of Nepal, has an IOS and Android app for beginner pythonistasFrom “Hello World” to Generators and DecoratorsOriginal Owners of Programiz: Ranjit Bhatta, Aswin Shrestha and Punit JajodiaSwipe, Learn, and RepeatHas an editor built inLeaderboardsQuizzes and “Interview Challenges” (paid)Newest update with 200+ programsInteractive IDEFREE, with minimal app advertisingPaid version as wellLearn anywhere availability, helps to reduce the Digital Divide826 million learners online due to Covid1950% of those learners do not have a household computer706 million have no at home internetThe engineering behind the app and its editor function: https://www.programiz.com/blog/online-python-compiler-engineering/

    Brian #4: New packaging security funding & NYU

    Sumana Harihareswara“New York University – specifically Professor Justin Cappos – and I (Sumana) have successfully asked the US National Science Foundation for a grant to improve Python packaging security. The NSF is awarding NYU $800,000 over two years – from mid-2021 to mid-2023 – to further improve the pip dependency resolver and to integrate The Update Framework further into the packaging toolchain.”“NYU researchers and developers willFurther assess and improve pip’s dependency resolver, following up on the work done in 2020 and making ResolveLib more reusable by other tools in the packaging ecologySecure the PyPI-to-user pipeline by integrating TUF support for signed packages throughout PyPI’s clients (we’re targeting conda, pip, and bandersnatch initially)”TUF is The Update Framework (TUF) specification, From theupdateframework.io:“The Update Framework (TUF) helps developers maintain the security of software update systems, providing protection even against attackers that compromise the repository or signing keys. TUF provides a flexible framework and specification that developers can adopt into any software update system.ResolveLib “provides a Resolver class that includes dependency resolution logic.”. It’s an independent project, but vendored by pipbandersnatch “is a PyPI mirror client”

    Michael #5: Extra x8, hear all about it

    We on amazon and audible (so is Teaching Python)Sourcery added skip refactorings. I did major refactoring since, favorite:url = request.url if request.url else '' tourl = request.url or ''Via Matthew Feickert, it’s easy to become a PSF member and support the PSFBeanie adds indexes!Pycharm (and other JetBrains IDEs) in the browser via Dave Kirbylp.jetbrains.com/projectorgithub.com/JetBrains/projector-installerSQLite as a file format comes to Audacity (via Jon Bultmeyer)Prayson Daniel shared his Neo4J examples using Neomodel.Call for proposals for PyCon Latam just went live

    Sean #6: Using Development Containers with VS Code for Students

    One of the big chores at the start of a course is getting everyone set up in the same development environment. Usually takes at least a few classes and nearly impossible to get everyone with the same environment at the end.Solution - use Docker images and the Remote-Containers extension to allow students to run a pre-configured container right in VS Code.Needs VS Code & Docker Desktop (maybe a remote Docker host could make it even easier?)Can deploy directly from a Github repo with persistent storage volumes, default VS Code settings, and even pre-installed packages and environment variables.Seems well suited for universities, experienced learners, etc. Also makes it easy to create different environments by assignment or course unit. What if every Talk Python training course had a GitHub container repo with the same starting environment?

    Extras:

    Kelly:

    https://us.pycon.org/2021/summits/education-training/

    Sean:

    Repl.it adds Python package caching (and gets a 40% speed boost)Home Assistant (actually Nabu Casa) buys ESPHome (makes DIY IoT devices cheap & easy to integrate)It’s good to know a Python programmer if you need a vaccine appointment

    Joke (in honor of our teachers today):

  • Sponsored by Linode! pythonbytes.fm/linodeSpecial guest: Sebastian Witowski

    Watch on YouTube

    Brian #1: Raspberry Pi Pico

    Release AnnouncementA review$4 microcontrollerSmallExtremely low power needs.Built on RP2040, a brand-new chip developed by Raspberry PiRelated: Mu : codewith.mu, 1.1.0-beta.2Mu is “a simple Python editor for beginner programmers.”1.1.0 support new boards, including Pico, Lego Spike, plus lots of new fixes.

    Michael #2: New MongoDB ODM: Beanie

    via PyCodersBeanie - is an asynchronous ODM for MongoDB, based on Motor and Pydantic.Very new but also very exciting.Main component of Beanie is Pydantic. It helps to implement the main feature - data structuring. Beanie Document - is an abstraction over the Pydantic BaseModel that allows working with Python objects at the application level and JSON objects at the database level.Example, classes: class TagColors(str, Enum): RED = "RED" BLUE = "BLUE" GREEN = "GREEN" class Tag(BaseModel): name: str color: TagColors = TagColors.BLUE class Note(Document): # This is the document structure title: str text: Optional[str] tag_list: List[Tag] = []

    Sebastian #3: Sourcery

    No, not the Terry Pratchett novel (although this one is pretty cool too!)Gives you refactoring recommendations in your code editorIntegrates with PyCharm and VS CodeSuper easy to use - you get suggestions as you type and with one click you can apply themFree to use in the code editor (you will need a personal token) and paid plans with analytics, CI integration, etc.It keeps finding errors in my code (well, maybe I'm just a bad programmer 😉)

    Michael #4: Neomodel

    An Object Graph Mapper (OGM) for the neo4j graph database, built on the awesome neo4j_driverFeatures:Familiar Django model style definitions.Powerful query API.Enforce your schema through cardinality restrictions.Full transaction support.Thread safe.pre/post save/delete hooks.Django integration via django_neomodelExample of classes from neomodel import (config, StructuredNode, StringProperty, IntegerProperty, UniqueIdProperty, RelationshipTo) config.DATABASE_URL = 'bolt://neo4j:password@localhost:7687' class Country(StructuredNode): code = StringProperty(unique_index=True, required=True) class Person(StructuredNode): uid = UniqueIdProperty() name = StringProperty(unique_index=True) age = IntegerProperty(index=True, default=0) # traverse outgoing IS_FROM relations, inflate to Country objects country = RelationshipTo(Country, 'IS_FROM')Relationships germany = Country.nodes.filter(code='DE) jim = Person.nodes.get(name='Jim') jim.country.connect(germany) if jim.country.is_connected(germany): print("Jim's from Germany") for p in germany.inhabitant.all(): print(p.name) # Jim, ... len(germany.inhabitant) # N: int # Find people called 'Jim' in germany germany.inhabitant.search(name='Jim') # Find all the people called in germany except 'Jim' germany.inhabitant.exclude(name='Jim')

    Brian 5#: A mock must always have a spec

    From Stargirl Flowers “My Python testing style guide”Great guide altogether, but this bit about mock specs is awesome.Some mocking guidance:Use real objects for collaborators whenever possibleA mock must always have a spec with mock.create_autospec() or mock.patch(..., autospec=True).“This ensures there's some connection between your mock and the real collaborator's interface. If you change the collaborator's interface in a way that breaks downstream targets, those targets tests will rightfully fail.”Consider using a stub or fake (with examples)Consider a spy (real object + mock wrapper lets you assert called and such)Don't give mock/stubs/fakes special namesUse factory helpers to create complex collaboratorsAnd then some random weird advice:“Use fixtures sparingly” - Now them’s fighting words. :)

    Sebastian #6: Conference radar

    The PyCon 2021 Call for Proposal acceptance emails will be sent soon, so let's talk about conferences.It's 2021, and just like the last year, most conferences are moving to an online format.Which is great, because it's so much easier to attend them. Not only the tickets got cheaper or even free, but you also don't have to pay for the accommodation, plane tickets and you don't have to actually fly anywhere.But how do you find what are the upcoming conferences? There is a list of conferences at python.org, but it doesn't have the smaller, local events, and you don't immediately see when each conference is taking place.I've found a tool called "conference radar" - a Python package that gives you a CLI tool to check for upcoming Python conferences!It prints a nice ASCII table with the dates of each conference. There are even some options that you can pass, for example, to see which conferences have an open Call for Proposal, in case you want to submit something. The main downside is that plenty of conferences are not included there, but I hope that the list of sources will be expanded in the future. The CFP flag is also not working very well, I guess, because it's hard to parse the data sources and extract this information automatically.So far, my best way to stay on top of the open CFPs is to follow my friend Miroslav on Twitter.MK: Heads up on installation. They say you can pip install conrad but the actual command is pip install conference-radar

    Extras

    Michael

    Announcing Modern Python Projects course: talkpython.fm/modern-python-projectsNow highlighting live livestreams on Python Bytes: pythonbytes.fm/stream/liveMars again. Yes, Python IS on Mars. See tweet.Signups for the Python Language Summit at PyCon (online only) are now open.

    Sebastian

    I've started using VS Code in the browser (a new project for a new client), and it's surprisingly good! I would never try it myself (I love to have all my tools installed locally on my computer). The worst part? Learning to not click "Ctrl+W" when I want to close a tab in VS Code (as it closes the whole tab in my browser). I'm curious to see if it will ever become a standard in programming. It's definitely a great way to set up a standardized development platform for the whole team.

    Joke: He has WiFi

  • Special guest: Calvin Hendryx-Parker

    Live stream

    Watch on YouTube

    Michael #1: AWSimple

    by James Abel AWSimple is a more object oriented interface on top of boto3 for some of the common “serverless” AWS services: S3, DynamoDB, SNS, and SQS. Features:Simple Object Oriented API on top of boto3One-line S3 file write, read, and deleteAutomatic S3 retriesLocally cached S3 accessesTrue file hashing (SHA512) for S3 files (S3's etag is not a true file hash)DynamoDB full table scans (with local cache option)DynamoDB secondary indexesBuilt-in pagination (e.g. for DynamoDB table scans and queries). Always get everything you asked for.Can automatically set SQS timeouts based on runtime data (can also be user-specified)Caching: S3 objects and DynamoDB tables can be cached locally to reduce network traffic, minimize AWS costs, and potentially offer a speedup.

    Brian #2: coverage and installed packages

    I’ve covered coverage.py a lot on Test & Code, starting with episode 12, and even talked about it on episode 147, and many others.Except there’s something I missed, hidden in plain sight, all this time.coverage --source , as well as pytest --cov if using pytest-cov plugin, is not just a path.“You can specify source to measure with the --source command-line switch, or the [run] source configuration value. The value is a comma- or newline-separated list of directories *or package names*. If specified, only source inside these directories or packages will be measured.” - coverage.py docs, (emphasis mine)Up to now I was doing this trick I picked up from I don’t remember where I would run coverage from the top level project directory, specify the source as the project source, and set a [paths] setting in .coveragerc, the source setting to both the project source and the site-packages directory. Then the report would show the coverage of the source code, even though it was the site-packages code that was running.That trick is still nice to specify the output as your project directory, which is usually a shorter relative path.However, it’s not essential. You can just specify the source as the package name, without the above trick, and coverage will report the coverage of the installed package. That is usually good enough. Super cool

    Calvin #3: Finding Mona Lisa in the Game of Life with JAX

    by Atul VinayaLots of great code examplesShowcases the speed increase you can get using JAX on a GPU vs CPU unvectorizedInitial implementation took days of CPU time to get a rough resultJAX compiles numpy to highly vectorized code to run on a GPURequires some refactor of the code to optimize for a highly parallel run on GPUsPost includes link to notebook used for the project “Running ~1000 iterations for a 483px wide Mona Lisa on the google colab GPU runtime only takes around 40 seconds!”

    Michael #4: Python Package Index nukes 3,653 malicious libraries uploaded soon after security shortcoming highlighted

    From Mark LittleRecall Google’s Python goal was around PyPI security.Related (from @tonny) Poison packages – “Supply Chain Risks” user hits Python community with 4000 fake modulesPyPI has removed 3,653 malicious packages uploaded days after a security weakness in the use of private and public registries was highlighted.“Developers are often advised to review any code they import from an external library though that advice isn't always followed.” ← yeahLast month, security researcher Alex Birsan demonstrated how easy it is to take advantage of these systems through a form of typosquatting that exploited the interplay between public and private package registries.Birsan set out to see whether he could identify the names of private packages used inside companies and create malicious packages using those library names to place in the public package registries – the indexes that keep track of available software modules.The names of private packages turned out to be rather easy to find, particularly in the Node.js/JavaScript ecosystem because private package.json files show up rather often in public software repositories.So Biran crafted identically named libraries that he designed to sneak system configuration data through corporate firewalls.The challenge then became getting applications that require private libraries to look for those file names in a polluted public source. As it turns out, it's common for corporate software developers to rely on a hybrid configuration for their applications, one that references private internal packages but also supports fetching dependencies from a public registry, in order to ensure packages are up-to-date.The companies that Birsan managed to attack with this technique include Apple, Microsoft, Netflix, PayPal,Shopify, Tesla, Uber, and Yelp. And for his efforts, he has been awarded at least $130,000 from bug bounty programs involving these firms.Birsan's success in carrying out such attacks should set off alarm bells. Software supply chain attacks present a higher degree of risk than many threat scenarios because they have the potential to affect so many downstream victimsMakes me want to setup devpi + devpi-constrained just for internal projects.What to do?Don’t do mass bogus uploads like this to prove your point. We appreciate the message you are trying to deliver, but it’s already been documented so you are just making distracting work for other people who could more usefully be doing something else for the project.Don’t choose a PyPI package juat because the name looks right. Check that you really are downloading the right module from the right publisher. Even legitimate modules sometimes have names that clash, compete or confuse.Don’t hook internal projects to external repositories by mistake. If you are using Python packages that you haven’t published externally, then the one thing you can be sure of is that all external copies of “your” package are imposter modules, probably malware.Don’t blindly download package updates into your own development or build systems. Test and review everything you download before you approve it for use. Remember that packages typically include update-time scripts that run when you do the update, so malware infections could be delivered as part of the update process, not of the module source code that ultimately gets installed.

    Brian #5: python-adventure

    Brandon Rhodes"This is a faithful port of the “Adventure” game to Python 3 from the original 1977 FORTRAN code by Crowther and Woods (it is driven by the same advent.dat file!) that lets you explore Colossal Cave, where others have found fortunes in treasure and gold, though it is rumored that some who enter are never seen again. ““For extra authenticity, the output of the Adventure game in this mode, python3 -m adventure, is typed to your screen at 1200 baud.”“Colossal Cave Adventure is the first known work of interactive fiction and, as the first text adventure game, is considered the precursor for the adventure game genre. “ - wikipediarelated:Zork, 77-79, also an adventure game, was inspired by Collossal Cave Adventure, 75-77Zork on Chuck - Brian’s a Chuck fanBrandon, can we have Zork also?side note:Closest I got was Dungeons of Daggorath on TRS-80. Not text based. Early 80’s

    Calvin #6: Exciting New Features in Django 3.2

    From Haki BenitaUpcoming LTS Release in the 3 seriesExpected in AprilPost highlights some interesting new features that you might not have noticedNew FeaturesCovering Indexes in Postgres support (performance plus!)Timezones are hard and TruncDate now helps keep you from pulling out the foot cannonJSONObject DB Functions, helping the unstructured data world keep using PostgresSignal.send_robust() now logs exceptions so you don’t have to!The new QuerySet.alias() method allows creating reusable aliases for expressions (Performance!)The new display decorator makes creating calculated admin fields cleanerValue Expressions Detects Type, more cleaning up to allow the ORM to figure it outNotable missing feature is Async ORM, but this will be awesome when it landsMore are listed on the Django 3.2 Release Page

    Extras:

    Michael:

    Is Python on Mars? FastAPI Website course is out: talkpython.fm/fastapi-webAre you thinking of going to PyCon 2021? Over at Talk Python, we're giving away 5 tickets to the event: talkpython.fm/pycon2021Be sure to join us @ pythonbytes.fm/youtubeGot a chance to speak to the medical field about Python and programming superpowers on the Finding Genius podcast.

    Calvin:

    DjangoCon Europe 2021 CFP is Open until 4/1 https://2021.djangocon.eu/talks/cfp/Python Web Conf 20214 Tracks this year60 Amazing Speakers (almost 20% women)TicketsProfessional $199Student $99Grants Available!

    Joke:

    /** Logger */ private Logger logger = Logger.getLogger(); // This is black magic // from // *Some stackoverlow link // Don’t play with magic, it can BITE. # For the sins I am about to commit, may Guido van Rossum forgive me // Remove this if you wanna be fired } catch(Exception ex) { // Houston, we have a problem } int getRandomNumber() { Return 4; // chosen by fair dice roll. // guaranteed to be random. }

    https://twitter.com/LinuxHandbook/status/1368974401979383810

  • Sponsored by Datadog: pythonbytes.fm/datadog

    Special guest: Sebastián Ramírez

    Live stream

    Watch on YouTube

    Brian #1: Python Developers Survey 2020 Results

    Using Python for? Lots of reductions in percentages. Increases in Education, Desktop, Games, Mobile, and OtherPython 3 vs 294% Python3 vs 90% last yearPython 3.8 has 44% of Python 3 usage, 3.5 or lower down to 3%environment isolation54% virtualenv (I assume that includes venv)32% Docker 22% CondaWeb frameworks46% Flask43% Django12% FastAPI…2% Pyramid :(…Unit testing 49% pytest28% unittest13% mockOS68% Linux, 48% Windows, 29% Mac, 2% BSD, 1% otherCI: Gitlab, Jenkins, Travis, CircleCI … (Where’s GH Actions?)Editors: PyCharm, VS Code, Vim, …Lots of other great stuff in there

    Michael #2: Django Ninja - Fast Django REST Framework

    via Marcus Sharp and Adam Parkin (Codependent Codr) independentlyDjango Ninja is a web framework for building APIs with Django and Python 3.6+ type hints.This project was heavily inspired by FastAPI (developed by Sebastián Ramírez)Key features:Easy: Designed to be easy to use and intuitive.FAST execution: Very high performance thanks to Pydantic and async support.Fast to code: Type hints and automatic docs lets you focus only on business logic.Standards-based: Based on the open standards for APIs: OpenAPI (previously known as Swagger) and JSON Schema.Django friendly: (obviously) has good integration with the Django core and ORM.Production ready: Used by multiple companies on live projects.Benchmarks are interestingExample api = NinjaAPI() @api.get("/add") def add(request, a: int, b: int): return {"result": a + b}

    Sebastian #3: Pydantic 1.8

    Hypothesis plugin (for property-based testing).Support for [NamedTuple](https://pydantic-docs.helpmanual.io/usage/types/#namedtuple) and [TypedDict](https://pydantic-docs.helpmanual.io/usage/types/#typeddict) in models.Support for [Annotated](https://pydantic-docs.helpmanual.io/usage/schema/#typingannotated-fields) types, e.g.: def some_func(name: Annotated[str, Field(max_length=256)] = 'Bar'): pass

    Annotated makes default and required values more “correct” in terms of types. E.g. the editor won't assume that a function's parameter is optional because it has a default value of Field(``'``Bar``'``, max_length=256), this will be especially useful for FastAPI dependency functions that could be called directly in other places in the code.

    Michael #4: Google, Microsoft back Python and Rust programming languages

    Partially via Will ShanksGoogle and Microsoft join and strengthen forces with the foundations behind the Python and Rust programming languagesThe companies will get to help shape their future.Microsoft has joined Mozilla, AWS, Huawei and Google as founding members of the Rust Foundation.Google donated $350,000 to the Python Software Foundation (PSF), making the company the organization's first visionary sponsor.Google is investing in improved PyPI malware detection, better foundational Python tools and services, and hiring a CPython Developer-in-Residence for 2021.Other PSF sponsors include Salesforce, a sustainability sponsor contributing $90,000. Microsoft, Fastly, Bloomberg and Capital One are maintaining sponsors contributing $60,000 apiece.You’ll find Talk Python Training over at the PSF Sponsors as well.Microsoft has shown an interest in Rust, particularly for writing secure code: “Rust programming changes the game when it comes to writing safe systems software”Microsoft is forming a Rust programming team to contribute engineering efforts to the language's ecosystem, focusing on the compiler, core tooling, documentation and more.

    Brian #5: Semantic Versioning Will Not Save You

    Hynek SchlawackVersion numbers are usually 3 decimals separated by dots. SemVer is Major.Minor.MicroImplied promise is that if you depend on something and anything other than the Major version changes, your code won’t break.In practice, you have to be proactiveHave tests with good coveragePin your dependenciesRegularly try to update your dependencies and retestIf they pass, pin new versionsIf not, notify the maintainer of a bug or fix your codeBlock the versions that don’t workConsequences:ZeroVerVersion conflictsmayhemConsider CalVer

    Sebastian #6: OpenAPI 3.1.0

    It was released on February.Now the OpenAPI schemas are in sync and based on the latest version of JSON Schema. That improves compatibility with other tools. E.g. frontend components auto-generated from JSON Schema.Very small details to adjust in Pydantic and FastAPI, but they are actually more “strictly compatible” with OpenAPI 3.1.0, as they were made with the most recent JSON Schema available at the moment. The differences are mainly in one or two very specific corner cases.

    Note: OpenAPI 3.1.0 might not be Python-specific enough, so, in that case, I have an alternative topic: IDOM, which is more or less React in Python on the server with live syncing with the browser.

    Extras

    Michael

    Installing Python - training.talkpython.fm/installing-pythonboto3 types update (via Dean Langsam) - seems like boto type annotations is not maintained anymore, and the rabbit hole of github links sends you to *mypy_boto3_builder *(they have a gif example).Traverse up from the cwd to look for [HTML_REMOVED].venv[HTML_REMOVED] virtual environments #75 [](https://github.com/brettcannon/python-launcher/issues/75)[CLOSED](https://github.com/brettcannon/python-launcher/issues/75)[]Talk Python: AMA 2021 Episode

    Brian

    Thanks to Matthew Casari and NOAA for the great shirts.

    Joke

    More code comments jokes

    try { } finally { // should never happen } /* You may think you know what the following code does. * But you dont. Trust me. * Fiddle with it, and you'll spend many a sleepless * night cursing the moment you thought youd be clever * enough to "optimize" the code below. * Now close this file and go play with something else. */ const int TEN=10; // As if the value of 10 will fluctuate... // I am not responsible for this code. // They made me write it, against my will. // If this code works, it was written by Paul DiLascia. // If not, we don't know who wrote it options.BatchSize = 300; //Madness? THIS IS SPARTA!
  • Sponsored by Linode! pythonbytes.fm/linode

    Special guest: Greg Herrera

    YouTube live stream for viewers:

    Watch on YouTube

    Michael #1: boto type annotations

    via Michael Lernerboto3's services are created at runtimeIDEs aren't able to index its code in order to provide code completion or infer the type of these services or of the objects created by them. Type systems cannot verify themEven if it was able to do so, clients and service resources are created using a service agnostic factory method and are only identified by a string argument of that method.boto3_type_annotations defines stand in classes for the clients, service resources, paginators, and waiters provided by boto3's services.

    Example with “bare” boto3:

    Example with annotated boto3:

    Brian #2: How to have your code reviewer appreciate you

    By Michael LynchSuggested by Miłosz BednarzakActual title “How to Make Your Code Reviewer Fall in Love with You”but 🤮even has the words “your reviewer will literally fall in love with you.” literally → figuratively, pleaseTopic is important though, here are some good tips:Review your own code first“Don’t just check for mistakes — imagine reading the code for the first time. What might confuse you?”Write a clear change list description“A good change list description explains what the change achieves, at a high level, and why you’re making this change.”Narrowly scope changesSeparate functional and non-functional changesThis is tough, even for me, but important.Need to fix something, and the formatting is a nightmare and you feel you must blacken it. Do those things in two separate merge requests. Break up large change listsA ton to write about. Maybe it deserves 2-3 merges instead of 1. Respond graciously to critiquesIt can feel like a personal attack, but hopefully it’s not.Responding defensively will only make things works.

    Greg #3: REPODASH - Quality Metrics for Github repositories

    by Laurence MolloyDo you maintain a project codebase on Github?Would you like to be able to show the maturity of your project at a glance?Walk through the metrics availableUse-case

    Michael #4: Extra, extra, extra, extra, hear all about it

    Python 3 Float Security BugBuilding Python 3 from source now :-/ It’s still Python 3.8.5 on Ubuntu with the kernel patch just today! (Linux 5.4.0-66 / Ubuntu 20.04.2)Finally, I’m Dockering on my M1 mac via:docker context create remotedocker --docker "host=ssh://user@server"docker context use remotedocker

    docker run -it ubuntu:latest bash now works as usual but remotely!

    Why I keep complaining about merge thing on dependabot. Why!?! ;)

    Anthony Shaw wrote a bot to help alleviate this a bit. More on that later.

    Brian #5: testcontainers-python

    Suggested by Josh PeakWhy mock a database? Spin up a live one in a docker container. “Python port for testcontainers-java that allows using docker containers for functional and integration testing. Testcontainers-python provides capabilities to spin up docker containers (such as a database, Selenium web browser, or any other container) for testing.”import sqlalchemyfrom testcontainers.mysql import MySqlContainerwith MySqlContainer('mysql:5.7.32') as mysql: engine = sqlalchemy.create_engine(mysql.get_connection_url()) version, = engine.execute("select version()").fetchone() print(version) # 5.7.32The snippet above will spin up a MySql database in a container. The get_connection_url() convenience method returns a sqlalchemy compatible url we use to connect to the database and retrieve the database version.

    Greg #6: The Python Ecosystem is relentlessly improving price-performance every day

    Python is reaching top-of-mind for more and more business decision-makers because their technology teams are delivering solutions to the business with unprecedented price-performance. The business impact keeps getting better and better. What seems like heavy adoption throughout the economy is still a relatively small-inroad compared to what we’ll see in the future. It’s like water rapidly collecting behind a weak dam.It’s an exciting time to be in the Python world!

    Extras:

    Brian:

    Firefox 86 enhances cookie protectionsites can save cookies. but can’t share between sites. Firefox maintains separate cookie storage for each site.Momentary exceptions allowed for some non-tracking cross-site cookie uses, such as popular third party login providers.

    Joke:

    56 Funny Code Comments That People Actually Wrote: These are actually in a code base somewhere (a sampling):

    /* * Dear Maintainer * * Once you are done trying to ‘optimize’ this routine, * and you have realized what a terrible mistake that was, * please increment the following counter as a warning * to the next guy. * * total_hours_wasted_here = 73 */ // sometimes I believe compiler ignores all my comments // drunk, fix later // Magic. Do not touch. /*** Always returns true ***/ public boolean isAvailable() { return false; }
  • Sponsored by Datadog: pythonbytes.fm/datadog

    Special guest: Brett Cannon

    Watch on YouTube

    Brian #1: Keeping up with Rich

    Will McGugan has been building RichIt looks like it’s on its way to becoming a full fledged TUI (text user interface)December: Live view: no blog post on that, I don’t think.January: Tree view: Rendering a tree view in the terminal with Python and RichFebruary: Layouts: Building Rich terminal dashboardsfun fullscreen.py example, uses Live viewAlso, python -m rich will display a demo screen that shows tons of the stuff that Rich can doMany of the features also have a stand alone demo built in, like: $ python -m rich.layout $ python -m rich.tree $ python -m rich.liveAlthough I haven’t figured out how to kill the live demo. it doesn’t seem to time out, and it eats Ctrl-C in my terminal.I’d really like to use Rich for interactive stuff, like keyboard interrupts and arrow keys and tab and such. It’d be fun.Which brings me to the bottom right corner of the python -m rich output. It includes a GitHub Sponsor link for Will. Also, Will, unless it’s a contradiction to RTD TOS, I think you should include a Sponsor link in the Rich documentation. Let’s convince Will to make Rich a full TUI.

    Michael #2: 12 requests per second

    If you take a look around the blogosphere at various benchmarks for Python web frameworks, you might start to feel pretty bad about your own setup.The incredible work of the guys at magic stack, getting 100,000 requests per second from uvloop in a single thread.There’s the FastAPI benchmarksEven more mind-blowing is Japronto which claims an insane 1.2 million requests per-second in a single threadBut what about your “boring” Flask or Django app? And how realistic are these benchmarks? Usually, not very.Here’s an article diving into this for a “proper” ORM style app.12 - 80 requests per sec: Both our sync workers are now hitting a blazing 12 requests per second 😅 Using async workers seems to help a lot, but oddly Sanic struggles here.Be sure to run in prod on a “real” server setup (nginx + gunicorn or whatever)Compare this to Talk Python Training - Python 3, uWSGI, Pyramid, MongoDB, $20/mo serverGet around 125 requests/sec @ 100ms response time on a single server.More realistically, we can handle about 10,000-20,000 concurrent “realistic” users before performance suffers.

    Brett #3: Python Launcher for Unix reaches RC (probably 😉)

    Exclusive! 😁Started right after PyCon US 2018Implemented in Rust (it’s my “good size” Rust learning project)The Python Launcher for Windows works by:Checking the shebang lineIf VIRTUAL_ENV is setFind the newest pythonX.Y version/command on $PATHCan specify specific versions via e.g. -3 or -3.9PY_PYTHON and PY_PYTHON3 environment variables supported--list also worksCan use PYLAUNCH_DEBUG to see what the tool is doing--help covers all of thisUnix version differs in the preference of shebang versus VIRTUAL_ENV over shebangFigure on Unix you will chmod the executable bit if you truly care about the shebangI also assume at this point people will use entry points if they really want to tie something to an interpreter versionHow often do people peg their scripts to a specific Python version instead of python2 or python3?What do people think of this logic swap (hence the “probably”)?Unix bonus feature: will automatically use any virtual environment found in .venv in the working directory (and no, what directory is considered is not configurable 😁)All of this has made it py my preferred way of running Python on my machineReally useful with Starship and its Python support (does away with the big “Tip” box they have for the python_binary setting)

    Michael #4: Build a text editor with Python and curses

    [curses](https://docs.python.org/3/library/curses.html) is a library to avoid having to deal with low level issues like efficiently painting to the terminal screen and receiving user input.a barebones curses application import curses def main(stdscr): while True: k = stdscr.getkey() if k == "q": sys.exit(0) if __name__ == "__main__": curses.wrapper(main)Clear the screen with stdscr.erase()Adding text (a line of text) to the screen: stdscr.addstr(row, 0, line)The article covers interesting topics like a “view” into the file that fits the screen and a cursor you move around with arrow keys

    Brett #5: Pattern matching and accepting change in Python

    The “5-barrel foot-gun” in the room (to use Brian’s words from the last episode 😉)Usual places have people commenting from “I like this” to screaming bloody murderI think there are many “new” people to the language who don’t know Python prior to Python 3, so they don’t realize how much things used to regularly change in the languagePattern matching was very much debated publicly, so this wasn’t a secret (and I’m sorry if you didn’t have the time to participate, but Python development doesn’t even always wait for me and I’m on the SC, so …)The 2020 SC also announced publicly the possibility of this back in December with their recommendation to accept the PEP(s)Also usual comments of “why did they waste their time on that?!? Go fix packaging!” (and it’s almost always packaging 🙄)This is open source: some people wanted to put their personal time and effort into trying to get pattern matching into Python, so that’s what they didIf you want to help with Python’s packaging ecosystem, you can do so but trying to tell people what they “need” or “should” do with their time is simply rudeHistory repeats itself: every change is unwelcome unless it solves your problemPattern matching very much opens up opportunities for certain problems that were not easily possible before, e.g. parsers and compilers are classics (and hence why they are so often implemented in functional languages)I don’t think you will see this in nearly every code base like you do e.g. list comprehensionsE.g. I’m sure data scientists aren’t saying any of this since they got @, right? 😉People also claiming it isn’t Pythonic need to note that Guido helped drive thisDo you know what is Pythonic better than Guido? 😉He might not be BDFL anymore, but that doesn’t mean he still doesn’t have good design sense, i.e. if you like Python up to this point then trust Guido’s gut that this is a good thing“In Guido We Trust” (you can even get it on a mug 😉)If you use pattern matching in real-world code and have feedback to provide with enough time to consider it before b1, then please let python-dev knowE.g. there is a chance to change the meaning of _ if that is truly your biggest hang-upThis will all probably become a blog postRunning title is “The Social Contract of Open Source”These kinds of attitudes against people trying their best to make things better for folks is what led to Guido retiring from being the BDFL in the first place, me having to take a month off from open source every year, etc.Aside: more influenced by Scala than by Haskell (not sure where Michael and some other people I’ve seen online got the idea Haskell played into this)Did you know we got list comprehensions from Haskell?

    Brian #6: A Quick Intro to Structural Pattern Matching in Python

    aka the “switch” statement. I mean, the “match” statement.Also known as PEP 636, Appendix A — Quick Introcourtesy Guido van RossumThis finally helps me to get my head around simple uses of the new syntax for 3.10simple form: def http_error(status): match status: case 400: return "Bad request" case 401: return "Unauthorized" case 403: return "Forbidden" case 404: return "Not found" case 418: return "I'm a teapot" case _: # _ acts as a wildcard and never fails to match return "Something else"you can combine several matches using | case 401|403|404: return "Not allowed"Patterns can look like unpacking assignments, and can be used to bind variables: # The subject is an (x, y) tuple match point: case (0, 0): print("Origin") case (0, y): print(f"Y={y}") case (x, 0): print(f"X={x}") case (x, y): print(f"X={x}, Y={y}") case _: raise ValueError("Not a point")This is actually the tricky one. x and/or y can change values after this statement if their case is hit.More details in the article, er, appendix.It’s still going to take a while to get used to this, but this appendix is a good start.

    Joke

    true happiness
  • Sponsored by Datadog: pythonbytes.fm/datadog

    Special guest: Hannah Stepanek

    Watch on YouTube

    Michael #1: We Downloaded 10,000,000 Jupyter Notebooks From Github – This Is What We Learned

    by Alena Guzharina from JetBrainsUsed the hundreds of thousands of publicly accessible repos on GitHub to learn more about the current state of data science. I think it’s inspired by work showcased here on Talk Python.2 years ago there were 1,230,000 Jupyter Notebooks published on GitHub. By October 2020 this number had grown 8 times, and we were able to download 9,720,000 notebooks. 8x growth.Despite the rapid growth in popularity of R and Julia in recent years, Python still remains the most commonly used language for writing code in Jupyter Notebooks by an enormous margin.Python 2 went from 53% → 11% in the last two years.Interesting graphs about package usageNot all notebooks are story telling with code: 50% of notebooks contain fewer than 4 Markdown cells and more than 66 code cells.Although there are some outliers, like notebooks with more than 25,000 code lines, 95% of the notebooks contain less than 465 lines of code.

    Brian #2: pytest-pythonpath

    plugin for adding to the PYTHONPATH from the pytests.ini file before tests runMentioned briefly in episode 62 as a temporary stopgap until you set up a proper package install for your code. (cringing at my arrogance).Lots of projects are NOT packages. For example, applications.I’ve been working with more and more people to get started on testing and the first thing that often comes up is “My tests can’t see my code. Please fix.”Exampleproj/src/stuff_you_want_to_test.pyproj/tests/test_code.pyYou can’t import stuff_you_want_to_test.py from the proj/tests directory by default.The more I look at the problem, the more I appreciate the simplicity of pytest-pythonpathpytest-pythonpath does one thing I really care about:Add this to a pytest.ini file at the proj level: [pytest] python_paths = srcThat’s it. That’s all you have to do to fix the above problem.Paths relative to the directory that pytest.ini is in. Which should be a parent or grandparent of the tests directory.I really can’t think of a simpler way for people to get around this problem.

    Hannah #3: Thinking in Pandas

    Pandas dependency hierarchy (simplified):Pandas -> NumPy -> BLAS (Basic Linear Algebra Subprograms)Languages: - - Python -> C -> Assembly df["C"] = df["A"] + df["B"] A = [ 1 4 2 0 ] B = [ 3 2 5 1 ] C = [ 1 + 3 4 + 2 2 + 5 0 + 1 ]Pandas tries to get the best performance by running operations in parallel.You might think we could speed this problem up by doing something like this: Thread 1: 1 + 3 Thread 2: 4 + 2 Thread 3: 2 + 5 Thread 4: 0 + 1However, the GIL (Global Interpreter Lock) prevents us from achieving the performance improvement we are hoping for. Below is an example of a common threading problem and how a lock solves that problem.- Thread 1 total Thread 2 1 + 3 + 4 + 2 0 0 + 5 10 0 + 6 + 2 total += 10 0 13 total =10 0 total += 13 10 total = 13 13 Thread 1 total Thread 2 1 + 3 + 4 + 2 0 unlocked 0 + 5 10 0 unlocked + 6 + 2 total += 10 0 locked 13 total =10 0 locked 10 unlocked 10 locked total += 13 10 locked total = 13 23 unlockedAs it turns out, because Python manages memory for you every object in Python would be subject to these kinds of threading issues: a = 1 # reference count = 1 b = a # reference count = 2 del(b) # reference count = 1 del(a) # reference count = 0So, the GIL was invented to avoid this headache which only lets one thread run at a time.Certain parts of the Pandas dependency hierarchy are not subject to the GIL (simplified):Pandas -> NumPy -> BLAS (Basic Linear Algebra Subprograms)GIL -> no GIL -> hardware optimizationsSo we can get around the GIL in C land but what kind of optimizations does BLAS provide us with?Parallel operations inside the CPU via Vector registersA vector register is like a regular register but instead of holding one value it can hold multiple values.| 1 | 4 | 2 | 0 | + + + +| 3 | 2 | 5 | 1 | = = = = | 4 | 6 | 7 | 1 |Vector registers are only so large though, so the Dataframe is broken up into chunks and the vector operations are performed on each chunk.

    Michael #4: Quickle

    Fast. Benchmarks show it’s among the fastest serialization methods for Python.Safe. Unlike pickle, deserializing a user provided message doesn’t allow for arbitrary code execution.Flexible. Unlike msgpack or json, Quickle natively supports a wide range of Python builtin types.Versioning. Quickle supports “schema evolution”. Messages can be sent between clients with different schemas without error.Example >>> import quickle >>> data = quickle.dumps({"hello": "world"}) >>> quickle.loads(data) {'hello': 'world'}

    Brian #5: what(), why(), where(), explain(), more() from friendly-traceback console

    Do this: $ pip install friendly-friendly_traceback.install() $ python -i >>> import friendly_traceback >>> friendly_traceback.start_console() >>>Now, after an exception happens, you can ask questions about it. >>> pass = 1 Traceback (most recent call last): File "[HTML_REMOVED]", line 1 pass = 1 ^ SyntaxError: invalid syntax >>> what() SyntaxError: invalid syntax A `SyntaxError` occurs when Python cannot understand your code. >>> why() You were trying to assign a value to the Python keyword `pass`. This is not allowed. >>> where() Python could not understand the code in the file '[HTML_REMOVED]' beyond the location indicated by --> and ^. -->1: pass = 1 ^Cool for teaching or learning.

    Hannah #6: Bandit

    Bandit is a static analysis security tool.It’s like a linter but for security issues. pip install bandit bandit -r .I prefer to run it in a git pre-commit hook:# .pre-commit-config.yaml repos: repo: https://github.com/PyCQA/bandit rev: '1.7.6' hooks: - id: banditIt finds issues like:flask_debug_truerequest_with_no_cert_validationYou can ignore certain issues just like any other linter: assert len(foo) == 1 # nosec

    Extras:

    Brian:

    Meetups this week 2/3 done.NOAA Tuesday, Aberdeen this morning - “pytest Fixtures”PDX West tomorrow - Michael Presenting “Python Memory Deep Dive”Updated my training page, testandcode.com/trainingFeedback welcome. I really like working directly with teams and now that trainings can be virtual, a couple half days is super easy to do.

    Michael:

    PEP 634 -- Structural Pattern Matching: Specification accepted in 3.10PyCon registration openPython Web Conf reg openHour of code - minecraft

    Joke:

    Sent in via Michel Rogers-Vallée, Dan Bader, and Allan Mcelroy. :)

    PEP 8 Song

    Watch on YouTube

    By Leon Sandoy and team at Python Discord
  • Sponsored by us! Support our work through:

    Our courses at Talk Python Trainingpytest bookPatreon Supporters

    Special guest: Jennifer Stark - @_JAStark & guest on talkpython.fm/259

    Watch on YouTube

    Brian #1: Do you really need a virtualenv?

    Frost Ming doesn’t think so, based on the article You don't really need a virtualenvThe link slug is “introducing-pdm”, which I think would be a better title, but the first did work to get people to talk about it. Also, “Try PEP 582 today” may have been appropriate.Teaching new people is a problem:Telling them to first type python -m venv venvThen type source venv/bin/activate or . venv/bin/activateUnless you’re on windows, then type venv\scripts\activate.batThen type pip install -r requirements.txtYeah. It’s not pretty, not fun, and good luck not having anyone ask questions about why this is necessary.Also the Python version is specified in the venv. So if you upgrade Python versions, what happens to existing venvs? The article also discusses levels of venvs, and global tools that maybe you want not tied to each venv. But we have pipx for that, so I don’t think that’s a real issue.Enter PEP 582, still in draft mode.Instead of a venv directory, your project has a __pypackage__ directory. If you python -m pip install in your project directory, stuff just goes there instead of to the global Python.So it kinda acts like a venv for local packages, it just doesn’t include local copies of the Python executables, and such.This is probably a horrible description of 582, but oh well. Something like that.pdm supports 582 todayPDM stands for Python Development Master“It installs and manages packages in a similar way to npm that doesn't need to create a virtualenv at all!”Has a workflow that reminds me of Poetry, but doesn’t use a venv, uses a package directory instead.Conclusion:Huge props to Frost for this. It’s cool to see a tool that supports 582 and glimpse a possible Python future.However, this doesn’t solve the “teaching Python” problem. The setup is more complex than venv.I’m personally sticking with venv, well virtualenv, until (and if) 582 is supported by Python and pip.

    Michael #2: Copier - like cookiecutter

    A library for rendering project templates.Works with local paths and git URLs.Your project can include any file and Copier can dynamically replace values in any kind of text file.It generates a beautiful output and takes care of not overwrite existing files unless instructed to do so.To use as a CLI app: pipx install copierTo use as a library: pip install copierHas a simple Python APIMain advantage: Can update existing projectsRuns from basic YAML files

    Jennifer #3: Pandarallel - run pandas apply in parallel!

    simple install `pip install pandarallel [--upgrade] [--user]``import from pandarallel import pandarallelinitialise pandarallel.initialize(), set progress bar BOOL, set number of workers … (defaults to all cores)just use parallel_apply where you’d usually put apply

    Brian #4: Stop Using Print to Debug in Python. Use icecream Instead

    Khuyen Tranprint(f``"``{x=}``"``) is better than print(f``"``x: {x}``"``) but it’s still a lot of typing.With icecream, you can type ic(x) insted and git this nice output: ic| x: 5It’s less typing and just as nice. There’s more.ic() with no arguments logs the file, function, line number when it’s hit. Easy program flow tracing without a debugger.You can configure it to do this cool context thing even if you do pass in a value to print.You can configure custom prefix formatting with a callback function, so you can include the time or the user that’s logged in, or whatever else state you want to track.Since all output is prefixed with ic|, you can see it easilyWrites to stderr by default, so it doesn’t muck up stdout stuffClean it out of your code by searching for ic() statements. If you have normal print statements in your code, you don’t want to use print for debugging also.

    Michael #5: HTMX: Dynamic and live HTML without JavaScript

    htmx allows you to access AJAX, CSS Transitions, WebSockets and Server Sent Events directly in HTMBest seen via the examples section - try some out live on their siteHas a cool Server Requests pane for seeing what’s happening in the example

    Jennifer #6: PyLDAvis - Interactive Topic Model Visualisation

    Port of LDAvis R package (does this mean PyLDAvis is a wrapper? A translation?) by Carson Sievert and Kenny ShirleyUser calls pyLDAvis with fitted model made with your favourite library (eg Gensim, sklearn, GraphLab)Outputs include:term frequency within topic bar chartterm frequency within whole corpus bar chartnext to each bar is a word. You hover over the word and the topic circles adjust size to reflect representation of that term in that topic. topic circles - one for each topic, whose areas are setto be proportional to the proportions of the topics across the N total tokens in the corpusterm-topic circles, with area proportional to the frequencies with which a given term is estimated to have been generated by the topics of whole corpusslider to adjust relevance metric (0 = terms very specific to currently selected topic; 1 = terms frequently seen in many topics).

    Extras:

    Brian:

    I’m also speaking to a group of NOAA people next week.I’m speaking the Aberdeen Python User Group on the 10th of Feb. It’s virtual, so everyone can come.Excited about both. My kids are more impressed with the NOAA thing. It’s fun to impress your kids.

    Michael:

    Jet Brain’s fifth annual Developer Ecosystem survey

    Joke:

    Engineer helping a designer

    https://twitter.com/EduardoOrochena/status/1306944019268861953

  • Sponsored by Datadog: pythonbytes.fm/datadog

    Special guest: Jeremy Tanner

    Watch on YouTube

    Brian #1: Constant Folding in Python

    Arpit BhayaniConstant Folding is when a language replaces constant expressions at compile time rather than computing them at runtime.CPython does this while creating the bytecode.We can use dis to see it in action >>> import dis >>> dis.dis("day_sec = 24 * 60 * 60") 1 0 LOAD_CONST 0 (86400) 2 STORE_NAME 0 (day_sec) 4 LOAD_CONST 1 (None) 6 RETURN_VALUEPython tries to fold lots of constants, but not all.Seems to be based on size >>> x = 2 ** 64 # folded >>> x = 4 ** 64 # not folded >>> a = "-" * 4096 # folded >>> a = "-" * 4097 # not foldedDiscussion continues with a discussion of how CPython folding is implemented recursively and elegantly.Key takeaway for me: Remember to not try to pre-compute constants while coding. Make them easy to read and Python will handle the optimization for you.

    Michael #2: Update All Packages With pip-review

    via PyCodersUpdating Python packages can be a hassle.There are many of them - it's hard to keep track of all the newest versionsWhen you decide what to update, you still have to update each of them manually.Originally a part of the pip-tools package, it now lives on as a standalone convenience wrapper around pip.Usage (venv) $ pip install pip-review (venv)$ pip-review scikit-learn==0.23.2 is available (you have 0.23.1) scipy==1.5.4 is available (you have 1.4.1) seaborn==0.11.0 is available (you have 0.10.1) ...Once you've identified if you'd like to update your packages, you can update them all, automatically, using pip-review --autoLimit with constraints.txt fileCan do it interactively via pip-review --interactive

    Jeremy #3: Quantum Mechanical Keyboard Firmware

    How does the Python get into your computer? Much of it, through the keyboardWhy mechanical keyboards?ErgonomicsWhere is QWERTY from?And when?Keymapsand because it’s essentially an escape room built of hardware and open source software and dodgy international transactionsI’ve fallen further down the mechanical keyboard rabbit holeRGBKB SolKeyboardio AtreusKeyboard firmware is mostly written in chttps://github.com/qmkBut there are Python portionshttps://github.com/qmk/qmk_cliCompiler helperAPIReportingSo the CLI is Python, where else is Python?When looking to improve your keymap, key logging w/ Python to process logs for heatmapshttps://github.com/qmk/qmk_firmware/issues/714https://github.com/algernon/ergodox-layout/blob/master/tools/log-to-heatmap.py

    Brian #4: Reinventing the Python Logo

    Carlo Occhiena interview with Jessica WilliamsonSome cool logo history

    Early logo

    Current, from 2006, designed by Tim Parklin

    “The logo is actually based on mayan representations of snakes which very often represent only the head and perhaps a short length of tail. The structure of the snake representations the natural coiling/nesting of a snake as seen side on.” - Tim Parklin

    Jessica Williamson proposed a new one in 2020:

    Then there’s the rest of the article talking about the design process, etc.

    But….. just wanted to throw this out there. I’m happy with the 2006 version. - BrianMK: Have you ever seen the logos on the app stores?

    Michael #5: Private PyPI with Serverless Computing

    Project: aws-lambda-pypiBrings together a few pieces so you can easily run a small PyPICloud instance as an AWS Lambda function with the help of Terraform.PyPICloud lets you publish Python packages privately.AWS Lambda lets you run a small service for free.Terraform ensures the service is deployed and maintained correctly.Security: This project takes the following security precautions.The session keys are auto-generated and stored in a secret.The server configuration file, which contains the session keys, is generated on the fly to minimize the possibility of exposure.The lambda function runs with a role limited to accessing only what it needs.PyPICloud is configured to display no packages to unauthenticated users.

    Jeremy #6: Beyond the Basic Stuff w/Python

    Al SweigartWant to become less feral?

    Extras:

    Jeremy:

    Twttier Haiku bot: @hitch_haiku

    Michael:

    Django 3.2 alpha 1 releasedmypy 0.800 with Python 3.9 Supportpip 21.0 out and it drops Python 2Elastic changes open-source license to monetize cloud-service use

    Brian:

    We talked about pip-chill in episode 208 pip-chill now has a --no-chill option to not list itself. Nice.I was just on https://www.twitch.tv/microsoftdeveloper, short, but fun

    Joke

    via Wolf

    by Kat Maddox

    developer: so i have good news and bad news

    manager: what's the good news?developer: i've discovered that the "5 second rule" only applies to foodmanager: and the bad news?developer: i dropped our tables
  • Sponsored by Linode! pythonbytes.fm/linode

    Special guest: Ogi Moore

    Watch the live stream on YouTube.

    Michael #1: diskcache

    via Ian MaurerPython disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.The cloud-based computing of 2020 puts a premium on memory. Gigabytes of empty space is left on disks as processes vie for memory. Among these processes is Memcached (and sometimes Redis) which is used as a cache. Wouldn't it be nice to leverage empty disk space for caching?Features:Pure-PythonFully DocumentedBenchmark comparisons (alternatives, Django cache backends)100% test coverageHours of stress testingPerformance mattersDjango compatible APIThread-safe and process-safeSupports multiple eviction policies (LRU and LFU included)Keys support "tag" metadata and evictionDeveloped on Python 3.8Tested on CPython 3.5, 3.6, 3.7, 3.8Tested on Linux, Mac OS X, and WindowsTested using Travis CI and AppVeyor CI

    Brian #2: TOML is 1.0.0 now.

    What does that mean for Python?Hopefully, some kind of toml parser will make it into Python core. Any Python access to 1.0.0? MixedImplementations and TOML version support page lists:pytomlpp supports 1.0.0-rc.3, which is a wrapper around C++ tomlplusplus, which does support 1.0.0. Confusingtomlkit supports 1.0.0-rc.1, so that’s promisingtoml supports 0.5.0, great name. It’d be cool if it would support 1.0.0What’s different between 0.5.0 and 1.0.0?Unless I’m mistaken, not much: CHANGELOG1.0.0-rc1Leading zeroes in exponent parts of floats are permitted.Allow raw tab characters in basic strings and multi-line basic strings.Allow heterogenous values in arrays.Other than that, lots of “Clarify …”, which I’m not sure how those all affect implementation.I’d love to hear more from people who know more about this

    Ogi #3: pyqtgraph

    pyqtgraph - plotting library, for when you need fast/interactive plotsUses qt5 (and soon qt6) bindings to generate plots within Qt applicationsFills a niche role, want easy mouse interactivity, running locally on a machineOften used in engineering/scientific applications when looking at a lot of data, and wanting interactivity

    Michael #4: Parler + Python = Insurrection in public

    via Jim Kring and Mark LittleAccording to Wikipedia: Parler (/ˈpɑːrlər/) is an American alt-tech microblogging and social networking service. Parler has a significant user base of Donald Trump supporters, conservatives, conspiracy theorists, and right-wing extremists.ArsTechnica article send in by Mark LittleArs: Parler’s amateur coding could come back to haunt Capitol Hill riotersCoding messA key reason for her success: Parler’s site was a mess. Its public API used no authentication. When users deleted their posts, the site failed to remove the content and instead only added a delete flag to it. Oh, and each post carried a numerical ID that was incremented from the ID of the most recently published one.Another amateur mistake was Parler’s failure to scrub geolocations from images and videos posted online.Some 80 terabytes of posts, 1M videos, many already deleted, preserved for posterity.Catalog and Python pointed out by Shaun King.See the catalog (maybe, it’s the ugly side of people).The gist: https://gist.github.com/kylemcdonald/d8884da1a82ef50754ee49e0b6561071Partially back online with Russian hosting service?

    Brian #5: Best-of Web Development with Python

    Suggested by Douglas NicholsCool list with nice iconsCoversFrameworks, HTTP Clients, ServersAuth tools, HTML Processing, URL utilitiesOpenAPI, GraphQL, WebsocketRPC, Serverless, Content ManagementWeb Testing, Web Forms, MarkdownThird-party APIsEmail, Web Scraping & Crawling, MonitoringAdmin UIAPI ProxiesFlask/FastAPI/Pyramid/Django UtilitiesNice to see lots of FastAPI projects:fastapi-sqlalchemy - Adds simple SQLAlchemy support to FastAPI. fastapi-plugins - FastAPI framework plugins. fastapi_contrib - Opinionated set of utilities on top of FastAPI. starlette_exporter - Prometheus exporter for Starlette and FastAPI. fastapi-utils - Reusable utilities for FastAPI. fastapi-code-generator - This code generator creates FastAPI app from an.. slowapi - A rate limiter for Starlette and FastAPI. fastapi-versioning - api versioning for fastapi web applications. fastapi-react - Cookiecutter Template for FastAPI + React Projects. Using.. fastapi_cache - FastAPI simple cache.

    Ogi #6: Assorted

    Pyjion - https://github.com/tonybaloney/Pyjion a JIT extension for CPython that compiles python code using .NET 5 CLR CuPy - NumPy compatible multi-dimensional array on CUDA, uses _``*array_function_* (enabled with numpy 1.17) code using numpy to operate directly on CuPy arrayssee NEP-18 and CuPy docscompatible with other libraries as well

    Extras:

    Michael:

    Trying Firefox + Brave + VPNPython Web Conf 2021 call for talks, due Jan 29, I’ll be speaking!PyCon US 2021 launched call for proposals:December 22, 2020 — Call for proposals openedFebruary 12, 2021 — Proposals are dueMarch 16, 2021 — Notifications will be sent to presentersMarch 23, 2021 — Deadline for speakers to confirm participationMarch 30, 2021 — Schedule is publicly releasedApril 28, 2021 — Deadline to submit pre-recorded presentation (tutorials will be live)May 12-13, 2021 — Tutorial daysMay 15-16, 2021 — Conference daysApple launching Racial Equity and Justice Initiatives with partners across a broad range of industries and backgrounds — from students to teachers, developers to entrepreneurs, and community organizers to justice advocates

    Brian:

    PyCascades 2021 schedule https://2021.pycascades.com/program/schedule/

    Ogi:

    Anthony Explains Video SeriesLearn X in Y minutesReading Working in Public by Nadia Eghbal - provides some sanity checks for existing maintainers, might be a fantastic perspective for new contributors to open source

    Joke

    Tech Support, 2x

    Working at the help desk? Get the theme song:

    Here to help song

    And help by chat:

    "Running a successful open source project is just Good Will Hunting in reverse, where you start out as a respected genius and end up being a janitor who gets into fights." - Byrne Hobart
  • Sponsored by Datadog: pythonbytes.fm/datadog

    Special guest: Jousef Murad, Engineered Mind podcast (audio, video)

    Watch on YouTube

    Brian #1: pip search. Just don’t.

    pip search [query] is supposed to “Search for PyPI packages whose name or summary contains [query]”The search feature looks like it’s going to be removed and the PyPI api for it removed.Alternative, and better approach, just manually look at pypi.org and search for stuff. Right now it does this: $ pip search pytest ERROR: Exception: Traceback (most recent call last): ... [longish traceback ommited] --- xmlrpc.client.Fault: [Fault -32500: "RuntimeError: PyPI's XMLRPC API has been temporarily disabled due to unmanageable load and will be deprecated in the near future. See https://status.python.org/ for more information."]The Python Infrastructure status page says, as of Jan 12: “Update - The XMLRPC Search endpoint remains disabled due to ongoing request volume. As of this update, there has been no reduction in inbound traffic to the endpoint from abusive IPs and we are unable to re-enable the endpoint, as it would immediately cause PyPI service to degrade again.”This started becoming a problem in mid December.The endpoint was just never architected to handle the scale it’s getting now. There’s a current issue “Remove the pip search command”, open on pip.The commend thread is locked now, but you can read some of the history.I personally don’t understand the need to hammer search with a CI system or other. Probably should be using a local cache or local pypi mirror for an active/aggressive CI system.If you have scripts or jobs that run pip search , it ain’t gonna work, so probably best to remove that.

    Michael #2: QPython - Scripting for Android with Python

    Python REPL on Android - interestingScripting Android tasks with Python - more interestingFree, open source app that is ad supported.Some people have commented that their phone is their only “computer”With SL4A features, you can use Python programming to control Android work:Android Apps API, such as: Application, Activity, Intent & startActivity, SendBroadcast, PackageVersion, System, Toast, Notify, Settings, Preferences, GUIAndroid Resources Manager, such as: Contact, Location, Phone, Sms, ToneGenerator, WakeLock, WifiLock, Clipboard, NetworkStatus, MediaPlayerThird App Integrations, such as: Barcode, Browser, SpeechRecongition, SendEmail, TextToSpeechHardwared Manager: Carmer, Sensor, Ringer & Media Volume, Screen Brightness, Battery, Bluetooth, SignalStrength, WebCam, Vibrate, NFC, USB

    Jousef #3: Thesis: Deep Learning assistant for designers/engineers

    PyTorch (3D) / TensorFlowThe thesis: what is it actually about & goal of the thesisLibraries mainly used: numpy, pandas(Reinforcement Learning & GANs)

    Brian #4: sortedcontainers

    Thanks to Fanchen Bao for the topic suggestion.Pure-Python, as fast as C-extensions, sorted collections library. >>> from sortedcontainers import SortedList >>> sl = SortedList(['e', 'a', 'c', 'd', 'b']) >>> sl SortedList(['a', 'b', 'c', 'd', 'e']) >>> sl *= 10_000_000 >>> sl.count('c') 10000000 >>> sl[-3:] ['e', 'e', 'e'] >>> from sortedcontainers import SortedDict >>> sd = SortedDict({'c': 3, 'a': 1, 'b': 2}) >>> sd SortedDict({'a': 1, 'b': 2, 'c': 3}) >>> sd.popitem(index=-1) ('c', 3) >>> from sortedcontainers import SortedSet >>> ss = SortedSet('abracadabra') >>> ss SortedSet(['a', 'b', 'c', 'd', 'r']) >>> ss.bisect_left('c') 2“All of the operations shown above run in faster than linear time.”Types:SortedListSortedKeyList (like SortedList, but you pass in a key function, similar to key in Pythons sorted function.)SortedDictSortedSetGreat documentation and tons of performance metrics in the docs.

    Michael #5: Łukasz Langa Typed Twitter Thread

    Let’s riff on typing for a bit. Here is my philosophy: If I have to type more than three characters to complete a symbol in my editor, something is wrong. e.g. to go from email_service. → email_service.send_account_email() I should only need to type .sae then tab/enter. These types of things are vastly better because of type hints.Python type hints are more malleable than even TypeScript.Lukasz is addressing this comment: Controversial take: Types in a Python code-base are a net negative.Pointsput enough annotations and tooling connects the dots, making plenty of errors evident.The most common to me at least is when a None creeps in. The second bug often caught by type checkers is on the "return" boundary: one of your code paths forgets a return.squiggly lines in your editorMicrosoft is now developing powerful type checking and code completion for Python in VSCode. This effort employs a member of the Python Steering Council, and possibly also the creator of Python himself soon. You think they would settle for "illusion of productivity"?

    Jousef #6:

    Point Cloud operations → open3d

    Extras:

    Michael:

    via Francisco Giordano Silva: On Brian's ref to using numpy all for array element-wise comparison, also please check out numpy.allclose method. Allows you to compare two arrays based on a given tolerance.

    Brian:

    Just this: 2021 is exhausting so far.Test & Code has shifted to every other week to allow time for other projects I’m working on.This is probably a short term change. But I don’t know for how long. It’s definitely not going away though. Just slowing down a bit.

    Jousef: Engineered Mind podcast

  • Sponsored by us! Support our work through:

    Our courses at Talk Python Trainingpytest bookPatreon Supporters

    Special guest: Jason McDonald

    Watch on YouTube

    Michael #1: 5 ways I use code as an astrophysicist

    Video by Dr. Becky (i.e. Dr Becky Smethurst, an astrophysicist at the University of Oxford)She has a great YouTube channel to check out.#1: Image Processing (of galaxies from telescopes)Noise removal#2: Data analysisImage features (brightness, etc)One example: 600k “rows” of galaxy properties#3: Model fittinge.g. linear fit (visually as well through jupyter)e.g. Galaxies and their black holes grow in mass togetherColor of galaxies & relative star formation#4: Data visualization#5: SimulationsBeautiful example of galaxies collidingStar meets black hole

    Brian #2: A Visual Intro to NumPy and Data Representation

    Jay AlammarI’ve started using numpy more frequently in my own work.Problem: I think of np.array like a Python list. But that’s not right.This visualization guide helped me think of them differently.Covers:arrayscreating arrays (I didn’t know about np.ones(), np.zeros(), or np.random.random(), so cool)array arithmeticindexing and slicingaggregation with min, max, sum, mean, prod, etc.matrices : 2D arraysmatrix arithmeticdot product (with visuals, it takes seconds to understand)matrix indexing and slicingmatrix aggregation (both all entries and column or row with axis parameter)transposing and reshapingndarray: n-dimensional arraystransforming mathematical formulas to numpy syntaxdata representationAll with excellent drawings to help visualize the concept.

    Jason #3: Qt 6 release (including PySide2)

    Qt 6.0 released on December 8: https://www.qt.io/blog/qt-6.0-released3D Graphics abstraction layer called RHI (Rendering Hardware Interface), eliminating hard dependency on OpenGL, and adding support for DirectX, Vulkan, and Metal. Uses native 3D graphics on each device by default.Property bindings: https://www.qt.io/blog/property-bindings-in-qt-6A bunch of refactoring to improve performance.QtQuick stylingCAUTION: Many Qt 5 add-ons not yet supported!! They plan to support by 6.2 (end of September 2021).Pay attention to your 5.15 deprecation warnings; those things have now been removed in 6.0.PySide6/Shiboken6 released December 10: https://www.qt.io/blog/qt-for-python-6-releasedNew minimum version is Python 3.6, supported up to 3.9.Uses properties instead of (icky) getters/setters now. (Combine with snake_case support from 5.15.2) from __feature__ import snake_case, true_propertyPyQt6 also just released, if you prefer Riverbank’s flavor. (I prefer official.)

    Michael #4: Is your GC hyper active? Tame it!

    Let’s think about gc.get_threshold().Returns (700, 10, 10) by default. That’s read roughly as:For every net 700 allocations of a collection type, a gen 0 collection runsFor every gen 0 collection run, 1/10 times it’ll be upgraded to gen 1.For every gen 1 collection run, 1/10 times it’ll be upgraded to gen 2. Aka for every 100 gen 0 it’s upgraded to gen 2.Now consider this: query = PageView.objects(created__gte=yesterday).all() data = list(query) # len(data) = 1,500That’s multiple GC runs. We’ve allocated at least 1,500 custom objects. Yet never ever will any be garbage.But we can adjust this. Observe with gc.set_debug(gc.DEBUG_STATS) and consider this ONCE at startup: # Clean up what might be garbage gc.collect(2) # Exclude current items from future GC. gc.freeze() allocs, gen1, gen2 = gc.get_threshold() allocs = 50_000 # Start the GC sequence every 10K not 700 class allocations. gc.set_threshold(allocs, gen1, gen2) print(f"GC threshold set to: {allocs:,}, {gen1}, {gen2}.")May be better, may be worse. But our pytest integration tests over at Talk Python Training run 10-12% faster and are a decent stand in for overall speed perf.Our sitemap was doing 77 GCs for a single page view (77!), now it’s 1-2.

    Brian #5: Top 10 Python libraries of 2020

    tryolabscriteriaThey were launched or popularized in 2020.They are well maintained and have been since their launch date.They are outright cool, and you should check them out.

    General interest:

    Typer : FastAPI for CLI applicationsI can’t believe first commit was right before 2020. Just about a year after the introduction of FastAPI, if you can believe it.Sebastián Ramírez is on 🔥 Rich : rich text and beautiful formatting in the terminal.Will McGuganyep. showed up in 2020, amazing.Dear PyGui : Python port of the popular Dear ImGui C++ project.PrettyErrors : transforms stack traces into color coded, well spaced, easier to read stack traces.Diagrams : lets you draw the cloud system architecture without any design tools, directly in Python code.

    Machine Learning:

    Hydra and OmegaConfPyTorch LightningHummingbirdHiPlot : plotting high dimensional data

    Also general

    Scalene : CPU and memory profiler for Python scripts capable of correctly handling multi-threaded code and distinguishing between time spent running Python vs. native code, without having to modify your code to use it.

    Jason #6: Adoption of pyproject.toml — why is this so darned controversial?

    The goal of this file is to have a single standard place for all Python tool configurations. It was introduced in PEP 518, but the community seems divided over standardizing.

    A bunch of tools are lagging behind others in implementing. Tracked in this repo

    A few of the bigger “sticking points”:

    setuptools is working on it: https://github.com/pypa/setuptools/issues/1688MyPy: GVR says it “doesn’t solve anything” and closed the PR. https://github.com/python/mypy/issues/5205Flake8 objections: https://gitlab.com/pycqa/flake8/-/issues/428#note_251982786Lack of standard TOML parser.“pip to change its behavior so mere presence of the file does not change functionality”Flake9 already implemented what Flake8 wouldn’t. Is this political?Bandit is sitting on a PR since 2019: https://github.com/PyCQA/bandit/issues/550ReadTheDocs: It’s too much work? — https://github.com/readthedocs/readthedocs.org/issues/7065PyOxidizer (shockingly), silent on the topic since 2019: https://github.com/indygreg/PyOxidizer/issues/93

    Extras:

    Michael:

    PyXLL for Excel people, including Python Jupyter Notebooks in Excel.Django 3.1.5 ReleasedPython 3.10.0a4 Is Now Available for TestingSciPy 1.6.0 ReleasedM1 + PyCharm fast? Example.Flying solo with the M1 too - apparently 75% is shutdown time for my MBP!

    Joke

    “Why did the programmer always refuse to check his code into the repository? He was afraid to commit.”

  • 2020 will be one for the history books, won't it? I've put together a great group to look back on 2020 - from the Python perspective.

    Join Brian and Michael along with Cecil Phillip, Ines Montani, Jay Miller, Paul Everitt, Reuven Lerner, and Matt Harrison for a light-hearted and fun look back on the major Python events of 2020.

    Video version of this episode: Watch on YouTube

    GuestsCecil PhillipInes MontaniJay MillerPaul EverittReuven LernerMatt Harrison
  • Sponsored by us! Support our work through:

    Our courses at Talk Python TrainingTest & Code PodcastPatreon Supporters

    Special guest: Anthony Shaw

    Stream on YouTube as wellWatch on YouTube

    Michael #1: Django Ledger Project

    by Miguel SandaThe mission is to provide free and open source accounting software that could easily replace commercial alternatives like QuickBooks.Django Ledger supports:Chart of Accounts.Financial Statements (Income Statement & Balance Sheets).Automatic financial ratio & insight calculations.Multi tenancy.Hierarchical entity management.Self-contained Ledgers, Journal Entries & Transactions.Financial Activities Support (operational/financial/investing).Basic OFX & QFX file import.Bills & Invoices with optional progressible functionality.Basic navigational templates.Entity administration & entity manager support.Bank Accounts.Not quite ready for production.This project is actively looking for contributors. Any financial and/or accounting experience is a big plus. If you have prior accounting experience and want to contribute, don't hesitate to contact.Browse the screenshots on the github repo.

    Brian #2: Flask-Meld: simple JavaScript interactive features without all of the JavaScript.

    Flask-Meld article with exampleMichael Abrahamsen, @MikeAbrahamsenworking example, code for exampleIt seems really zippy. Definitely gonna try this.Similar project for Django: django-unicornAdam Hill, @adamghilltodo example

    Anthony #3: Bitwise operators in Python (RealPython)

    Article by Bartosz ZaczyńskiParticularly useful for the enum.IntFlag and enum.Flag classes in the stdlib since 3.7 >>> class Color(Flag): ... RED = auto() ... BLUE = auto() ... GREEN = auto() ... WHITE = RED | BLUE | GREEN

    Michael #4: Why should you use an ORM (Object Relational Mapper)?

    You may have heard you should use an ORM, but why?To get a better understanding of how ORMs work, it’s helpful to work through the kind of problems they can solve.Data modeled as classes is great but has some shortcomingsHow do you query, filter, sort it?How do you store it? Classes live in memory only and pickling isn’t ideal for many reasonsThere is also concurrency, transactional operations/atomiticityEnter the DBBut plain SQL is error prone (did you refactor that name? oops)Plain SQL has code written within another language (SQL within Python, tools help but still)Plain SQL has potential security issues (unless you use parameters)ORMsORMs are libraries that sit between the SQL database world and your object-oriented programming language world.For all intents and purposes, they are an abstraction layer that allows, among other things, for the translation of Python to SQL.A cycle is formedDefine our models in the Python world as objectsthe ORM layer translates those models to SQL statements with SQL types to create tables and columns. In our application, we instantiate objects from those models, which creates rows in the aforementioned tables. Finally, when we want those objects back, we use Python code to retrieve them, which triggers the ORM layer to create the necessary query to retrieve the rows from the database. Then, the ORM layer takes the resulting rows and maps them back to objects.And you get migrations

    Brian #5: sqlite-utils: a Python library and CLI tool for building SQLite databases

    Simon Willison, @simonw“sqlite-utils is a combination Python library and command-line tool I’ve been building over the past six months which aims to make creating new SQLite databases as quick and easy as possible.”CLI for sqliteRun queries and output JSON, CSV, or tablesanalyzing tablesinserting datacreating, dropping, transforming tablescreating indexessearching…Python API for using as a libraryway easier interaction with sqlite. import sqlite_utils db = sqlite_utils.Database("demo_database.db") # This line creates a "dogs" table if one does not already exist: db["dogs"].insert_all([ {"id": 1, "age": 4, "name": "Cleo"}, {"id": 2, "age": 2, "name": "Pancakes"} ], pk="id")sqlite-utils docs

    Anthony #6: Online conferences are not working for me. But this was a good talk,

    “What the struct?!” by Zachary Anglin at the Pyjamas conference.Explains the struct libraries macro-language for converting binary data structures into Python native types (and vice versa) https://docs.python.org/3/library/struct.html#format-charactersReally nice if you want to programmatically edit/read binary structure (combining with the bitwise operators).If you’re getting into this topic. Save-game hacking is an interesting place to start. I use SynalizeIT! /Synalysis as a binary data grammar explorer. It has some example save game formats https://www.synalysis.net/formats.xmlIts also great for hacking/CTF.

    https://youtu.be/QT_NAk_peHQ?t=7130

    Extras

    Brian:

    Resurrecting Python PDX West as a virtual lunchtime event.January is planned for January 14, 2021

    Michael:

    Did a FastAPI webcast people can check out.M1 life is getting better and better

    Anthony:

    From February 1 I’ll be the new Python Open Source advocate at Microsoft, working with Nina ZacharenkoGive us a quick update on Pyjion + .NET Core?

    Jokes:

    https://twitter.com/lk012/status/1334390836172378113

  • Sponsored by us! Support our work through:

    Our courses at Talk Python TrainingTest & Code PodcastPatreon Supporters

    Special guest: Shari Eskenas

    Catch the video edition live stream on YouTubeWatch on YouTube

    Be part of the episode by subscribing and “smashing that bell” over at pythonbytes.fm/youtube

    Brian #1: pytest 6.2 is out

    pytester fixture for plugin testing. Like testdir, but a better name, and uses pathlib.Path objects.verbose mode now shows the reason a test was skipped in the terminal line after the SKIPPED, XFAIL, or XPASSCan use monkeypatch as a context manager with .context() and it’s available both in test and fixture functions, but also in helper functions by using pytest.Monkeypatch.context(). import os from contextlib import contextmanager import pytest def test_foo_1(monkeypatch): with monkeypatch.context() as mp: mp.setenv("foo", "bar") assert os.getenv("foo") == "bar" @contextmanager def some_func(): with pytest.MonkeyPatch.context() as mp: mp.setenv("foo", "bar") yield def test_foo_2(): with some_func(): assert os.getenv("foo") == "bar"Lots of other goodies.related: pytest-check, my only released plugin for pytest, was updated to support pytest 6.x

    Michael #2: SQLite as a file format (like docx)

    via Jon BultmeyerAn SQLite database file with a defined schema often makes an excellent application file format. Here are a dozen reasons why this is so:Simplified Application Development. No new code is needed for reading or writing the application file.Single-File Documents. An SQLite database is contained in a single file, which is easily copied or moved or attached.High-Level Query Language. SQLite is a complete relational database engine, which means that the application can access content using high-level queries.Accessible Content. Information held in an SQLite database file is accessible using commonly available open-source command-line tools - tools that are installed by default on Mac and Linux systems and that are freely available as a self-contained EXE file on Windows.Cross-Platform. SQLite database files are portable between 32-bit and 64-bit machines and between big-endian and little-endian architectures and between any of the various flavors of Windows and Unix-like operating systems.Atomic Transactions. Writes to an SQLite database are atomic. They either happen completely or not at all, even during system crashes or power failures.Incremental And Continuous Updates. When writing to an SQLite database file, only those parts of the file that actually change are written out to disk. This makes the writing happen faster and saves wear on SSDs.Easily Extensible. As an application grows, new features can be added to an SQLite application file format simply by adding new tables to the schema or by adding new columns to existing tables. Adding columns or tables does not change the meaning of prior queries.Performance. In many cases, an SQLite application file format will be faster than a pile-of-files format or a custom format.Concurrent Use By Multiple Processes. SQLite automatically coordinates concurrent access to the same document from multiple threads and/or processes.Multiple Programming Languages. Though SQLite is itself written in ANSI-C, interfaces exist for just about every other programming language you can think of.Better Applications. If the application file format is an SQLite database, the complete documentation for that file format consists of the database schema, with perhaps a few extra words about what each table and column represents.

    Shari #3: A Day in Code: Python – A picture book written in code

    Brian #4: PythonLabs is now hosted by Azure. and “Yes, Barry, there is a PythonLabs”

    I can’t believe we haven’t covered this in the last 211 episodes.But it seems like good timing now.Now resides at azure.pythonlabs.comBy Tim Peters (originally posted on January 6th, 2004 to the PSF members list)Barry (I’m assuming Barry Warsaw) asked the question: “… what /is/ Pythonlabs now? Or /is/ there a Pythonlabs now? I dunno -- Guido owns the domain name which is probably the biggest claim to Pythonlabhood there is.”Tim replies with a very “Yes, Virginia, there is a Santa Clause”-esque answer:

    Snippets include:

    Barry, your little friends are wrong. They have been affected by the skepticism of a skeptical age. …Yes, Barry, there is a PythonLabs. It exists as certainly as love and generosity and devotion exist, … Alas! how dreary would be the world if there were no PythonLabs! It would be as dreary as if there were no Barrys. …Not believe in PythonLabs! You might as well not believe in fairies. … Nobody sees PythonLabs, but that is no sign that there is no PythonLabs. …A thousand years from now, Barry, nay 10 times 10,000 years from now, it will continue to make glad the heart of childhood.

    Michael #5: Extra, extra, extra, extra, extra, extra, hear all about it

    #1 Numpy version pinning via GriceJust catching up on Ep 208. Note in part 2 about the Numpy issue, folks can pin versions by platform with environment markers: numpy==1.19.3; platform_system == 'Windows' numpy==1.19.4; platform_system == 'Linux'#2 Stylesheet for PySide2 and PyQt5, this time looks like Material Design - via William Jamir Silva#3 Talk Python hits 20M downloads. Python Bytes is almost 6M too.#4 Pyramid 2.0 is coming.#5 Python 3.9.1 is out with 282 changes. Ships as a universal binary (Intel + M1) on macOS.#6 Python + Mac Mini + M1 video#7 Python Steering Council selected

    Shari #6: OpenMV

    Extras

    Brian:

    agrs, kwargs (quargs), and communitya silly discussion on twitterNocole Carlson: “Huge debate … about whether you say “quargs” or “keyword arguments” for “kwargs”. Obviously “quargs” is correct.”It never occurred to me to say “quargs”, but I like it.Vicki Boykis replied that she’s “… never said anything but “quargs”. This is like a parallel universe.”Some other amusing responses.This reminded me of a conversation I might have over beer at PyCon. or at a booth, or just standing around in the hallway.I miss so much the in person community. I’m grateful that little bits of it are intact on twitter.

    Joke:

    Pizza delivery fail

    Second joke:

    Why do you many developers use dark mode?Because bugs are attracted to light.
  • Sponsored by Techmeme Ride Home podcast: pythonbytes.fm/ride

    Special guest: Matthew Feickert

    Live streamed on YoutubeWatch on YouTube

    Brian #1: Introducing FARM Stack - FastAPI, React, and MongoDB

    Aaron BassetDescribes a todo CRUD application, available on github, to show the concepts in action.Animated gif showing how to use the FastAPI interactive documentation to understand the API in a browser.ShowsHow app connects to routes and endpoints.The run call with uvicorn to get an async even loop goingConnecting to a MongoDB databaseDefining models and how easy it is to set up a schema.Discusses routes and endpoints and how to hook up all the CRUD endpoints.The React bit is an application that changes as you modify the elements through the endpoint interactive session.Very cool way to work and get something running fast.

    Michael #2: py-applescript

    How would you like to automate your mac with Python rather than the dreadful AppleScript?py-applescript is an easy-to-use Python wrapper for NSAppleScript, allowing Python scripts to communicate with AppleScripts and AppleScriptable applications.Features:Scripts may be compiled from source or loaded from diskStandard 'run' handler and user-defined handlers can be invoked with or without argumentsArgument and result values are automatically converted between common Python types and their AppleScript equivalentsCompiled scripts are persistent: handlers can be called multiple times and top-level properties retain their stateAvoids any dependency on the legacy appscript library, flawed Scripting Bridge framework, or limited osascript executableThe applescript package exports four classes - AppleScript, ScriptError, AEType and AEEnum - plus one constant, kMissingValue.

    Matthew #3: airspeed velocity

    Are you developing a tool where performance is of individual functions is key? Do you want to be able to benchmark those metrics over every commit in your project with beautiful interactive visualizations?From the docs:> airspeed velocity (asv) is a tool for benchmarking Python packages over their lifetime. Runtime, memory consumption and even custom-computed values may be tracked. The results are displayed in an interactive web frontend that requires only a basic static webserver to hostDeveloped by a community of people that you’ll probably recognize if you hang around the SciPy and Jupyter ecosystem lead by Michael Droettboom and Pauli VirtanenThe docs include examples for benchmarks of NumPy, SciPy, and Astropy (example: SciPy)Example: SciPy’s interpolate.Interpolate1d.time_interpolate testProduces plot of timing performance across multiple parameterizations of the testClick on any node in the plot and get taken directly to the commit on GitHub! 🤯 Developed openly on GitHubUp on PyPI as asvpython -m pip install asvVery cute and very Pythonic joke in the test dashboards: Report title for library X is “airspeed velocity of an unladen X” 😛

    Brian #4: Parsecs experience with Hypothesis testing

    Parsec blog, Parsec is a client side encrypted file sharing service .Discussion of their experience with Hypothesis to test a 4 year old, large, asynchronous Python project.In particular, an algorithm around RAID5 redundancy.The test needs to check that:blocks can be split into chunksblocks can be rebuilt from chunksblocks can be rebuilt with one missing storage chunkGoes on to describe how they use a stateful mode of hypothesis.It’s a rather complex description. But very welcome.I’ve been waiting to hear how people are using Hypothesis for real world problems.Their recommendations for when to use Hypothesis:If you are writing a kind of encoder/decoder. Hypothesis is simply the perfect solution for you ;-)If you can write a simple oracle (or even better: you already have one available given you are reimplementing a protocol !)If the work is of type "hard to compute, simple to check"

    Michael #5: Amazon Web Services now offers Mac instances based on Mac mini; M1 coming later

    Amazon Web Services (AWS) is now offering Mac instances to developers who want to speed up building and testing of Mac, iOS, iPadOS, watchOS and tvOS apps.You no longer need a Mac and macOS to build iOS apps.The instances are based on physical Mac minis, and you get exclusive access to a machine while you are using it.Right now, Amazon is using Intel-based Mac minis, but the company says that it will have M1 Mac minis available in the first half of next year. Check out the blog post and the video below for more details.Good as a backstop until we get the newer faster M1 chips if you/your company is holding off.Although there's no virtualization in play here, the mac1.metal instances can be spun up and down nearly as rapidly, thanks to the AWS Nitro hardware management—which is invisible, from the customer's perspective.To someone who spins up a mac1.metal instance, the instance is for all intents and purposes a perfectly vanilla, brand-new Intel Mac mini.But there is also macstadium at 1/10th the price.

    Matthew #6: visidata

    Are you doing some data analysis and get given a new CSV file? Your first instinct is to load it up into pandas, right? Wouldn’t it be nice to drop in and explore at the command line though a bit to get the lay of the land and even do some preliminary analysis?From the docs:> Data science without the drudgery!> > VisiData is an interactive multitool for tabular data. It combines the clarity of a spreadsheet, the efficiency of the terminal, and the power of Python, into a lightweight utility which can handle millions of rows with easeSupports large number of file formats: 42Everything from CSV and JSON through JIRA, MySQL, and PNGDeveloped openly on GitHub by Saul PwansonOn PyPI as visidatapython -m pip install visidataThough as this is a Python application you probably going to want this installed regardless of the virtual environment you’re working in. Distributions of visidata are available through the package managers you’d expect: apt, (also nix, Guix but not yum?), Homebrew, and Conda-forge so can easily get to work on most Unix-based machines.Tested it with our trusty beloved tool pipx and that works too (thanks again Chad Smith!)python -m pip install pipxpipx ensurepathpipx install visidataSome pretty impressive live demo videos on the websitevisidata has come in quite handy for me on a frequent basis as I deal with JSON serializations of statistical models with hundreds of parameters. Having a way to explore those JSON files without getting visually overwhelmed is super nice!

    Extras

    Brian:

    PyCon US 2021

    Michael:

    PSF Fund RaiserTickets for PyCascades 2021 are now available on pretix. Our ticket prices are $10 for students, $20 for individuals, and $50 for professional attendees.Live streaming on YouTube!

    Matthew:

    Advent of Code 2020 has started. Happy coding! ❄️ If time (don’t want to hog or go over):Follow up from Episode 205:awkward v1.0.0rc1 is on PyPIpip install awkward should get what you ant by the time this show goes live (no more need for awkard0)Kudos to Python Bytes for making full transcripts of all the shows available for view on pythonbytes.fm — this helps be more inclusive of the deaf Python community (including Matthew’s good friend and coauthor)

    Joke:

    How to fix any computer - via theoatmean.com

  • The live stream recording on YouTube.

    Special guest: Jay Miller

    Sponsored by us! Support our work through:

    Our courses at Talk Python TrainingTest & Code PodcastPatreon Supporters

    Brian #1: Analyzing Kickstarter Campaigns with Python Data Science Tools

    Article title: “Kickstarter Projects — Do They Succeed?”Aditya PatkarUsing a Kaggle dataset of 378,661-ish projects up to 2018.Looks at using pandas data frames to explore the data.Using .describe() data frame method to learn a lot.Uses matplotlib and seaborn to analyze the data further.Odd statement that I’m not sure is straight faced or a really dry joke: “The data from 1970 seems to be bad or insignificant data.”Examples of using heat maps, line graphs, bar charts, to look at different aspects.Some results:35.64% of projects are successful (meaning goal hit)tech asks for the most for goals, and has the highest average per backer.Comics has the lowest pledged amount per backer average.Nice that you can use the techniques to ask your own questions of the data.

    Michael #2: GPU Accelerated Python for Machine Learning on Cross-Vendor Graphics Cards

    Building machine learning algorithms using the Vulkan Kompute Python FrameworkWhen you hear “CUDA”, that means Nvidia 🙂Uses Vulkan Kompute frameworkA large number of high profile (and new) machine learning frameworks such as Google’s Tensorflow, Facebook’s Pytorch, Tencent’s NCNN, Alibaba’s MNN —between others — have been adopting Vulkan as their core cross-vendor GPU computing SDK.As you can imagine, the Vulkan SDK provides very low-level C / C++ access to GPUs, which allows for very specialized optimizations.The main disadvantage is the verbosity involved, requiring 500–2000+ lines of C++ code to only get the base boilerplate required to even start writing the application logic.The Kompute Python package is built on top of the Vulkan SDK through optimized C++ bindings, which exposes Vulkan’s core computing capabilities. Kompute is the Python GPGPU framework.The main article talks through a couple of numerical computation examples.

    Jay #3: Adafruit PyPortal - CircuitPython Powered Internet Display

    Gift for the tinkering pythonistaCircuitPyUse it to make plenty of cool thingsScreen/speaker/Light Sensor Built-In

    Brian #4: Introduction to Linear Algebra for Applied Machine Learning with Python

    Pablo CaceresIntended as a reference and not a comprehensive review.Still, I very much appreciate it.Includes links to both free and paid resources to thoroughly learn linear algebraCoverssets, ordered pairs, relations, functions, vectorsmatriceslinear and affine mappingsmatrix decompositionUses numpy, pandas, and altair for examplesQuick (but useful) explanations of concepts, along with how to represent and do it with numpyI’m really just getting into it, but I’m enjoying it and this is the right level of handholding I needed.

    Michael #5: How many notebook frameworks? Many, and now +1 with Deepnote

    Deepnote is a new kind of data science notebook. Jupyter-compatible with real-time collaboration and running in the cloud. Free for individuals, paid for teams and companiesReal time collaboration is a key featureBuilt in versioning comingCode review in the notebook coming“View” your variables as a whole environmentBetter — real — autocompleteDashboards coming too

    Jay #6: imagekit.io

    image cdnstarted using imagekit on my own website and noticed faster load timesallows for some responsive “fanciness”Add BlursSmart CroppingPython API or URL-Schema

    Extras

    Michael:

    The Apple M1 mac mini wait continues. :)Talk Python To Me, pro editionPSF Fundraiser for the month of December: https://pythonbytes.fm/psf2020

    Jay:

    Elastic Community YouTube ChannelJust posted my lightning talk on looking at open data from the government.Upcoming interview on one of our newest clients - Eland which is python client to create pandas-like dataframes with elasticsearch datastores.My Podcast The PIT Show weekly insights from me on my developer journey and interviews with amazing folks in the tech space.Elastic Blog - Just posted my first Elastic Blog post Elastic Contributor Program: How to create a video tutorial

    Joke:

    via twitter.com/Spirix3/status/1330611989891207168

    Q: why can't SQL and NoSQL Developers date one other?A: because they don't agree on relationships.
  • Sponsored by us! Support our work through:

    Our courses at Talk Python TrainingTest & Code PodcastPatreon Supporters

    Michael #1: Running Python on .NET 5

    by Anthony ShawTalked about pyjion way back when on episode 49 with Brett Cannon..NET 5 was released on November 10, 2020. It is the cross-platform and open-source replacement of the .NET Core project and the .NET project that ran exclusively on Windows since the late 90’s. See the conference about it if you want to go deeper.Performance: I just saw a SO post about someone complaining their Python was 31x slower than C#.The most common way around this performance barrier is to compile Python extensions from C or using something like Cython..NET 5 CLR comes bundled with a performant JIT compiler (codenamed RyuJIT) that will compile .NETs IL into native machine instructions on Intel x86, x86-64, and ARM CPU architectures.Pyjion is a project to replace the core execution loop of CPython by transpiling CPython bytecode to ECMA CIL and then using the .NET 5 CLR to compile that into machine code.It then executes the machine-code compiled JIT frames at runtime instead of using the native execution loop of CPython.A few releases of Python ago (CPython specifically, the most commonly used version of Python) in 3.7 a new API was added to be able to swap out “frame execution” with a replacement implementation. This is otherwise known as PEP 523.This extension uses the same standard library as Python 3.9.Will this be compatible with my existing Python code? What about C Extensions?The short answer is- if your existing Python code runs on CPython 3.9 – yes it will be compatible.Tested against the full CPython “test suite” on all platforms. In fact, it was the first JIT ever to pass the test suite.Is this faster? The short answer a little, but not by much (yet).see also: https://twitter.com/anthonypjshaw/status/1328457723608928256?s=20

    Brian #2: PEP 621 -- Storing project metadata in pyproject.toml

    Progress on standardizing what goes into pyproject.tomlAuthors Brett Cannon, Paul Ganssle, Pradyun Gedam, Sébastien Eustace (of poetry), Thomas Kluyver (of flit), Tzu-Ping ChungMotivators of this PEP are:Encourage users to specify core metadata statically for speed, ease of specification, unambiguity, and deterministic consumption by build back-endsProvide a tool-agnostic way of specifying metadata for ease of learning and transitioning between build back-endsAllow for more code sharing between build back-ends for the "boring parts" of a project's metadataDoesn’t change any existing core metadataDoesn’t attempt to standardize all possible metadataIncluded in table named [project]:nameversiondescriptionreadmerequires-pythonlicenseauthors/maintainerskeywordsclassifiersurlsentry pointsdependencies/optional-dependenciesdynamic There’s an example in the PEP that helps clear things upMany items have synonyms specified for flit/poetry/setuptools (presumably for backward compatibility)

    Michael #3: GitHub revamps copyright takedown policy after restoring YouTube-dl

    In October following a DMCA complaint from the Recording Industry Association of America (RIAA) it was taken down at GitHub. Citing a letter from the Electronic Frontier Foundation (the EFF), GitHub says it ultimately found that the RIAA’s complaint didn’t have any merit. The RIAA argued the tool ran afoul of section 1201 of the US copyright law by giving people the means to circumvent YouTube’s DRM. the EFF dissects the RIAA’s claims, highlighting where the organization had either misinterpreted the law or how the code of YouTube-dl works. “Importantly, YouTube-dl does not decrypt video streams that are encrypted with commercial DRM technologies, such as Widevine, that are used by subscription videos sites, such as Netflix,” the organization points out when it comes to the RIAA’s primary claim.GitHub is implementing new policies to avoid a repeat of a repeat situation moving forward. First, it says a team of both technical and legal experts will manually evaluate every single section 1201 claim.If the company’s technical and legal teams ultimately find any issues with a project, GitHub will give its owners the chance to address those problems before it takes down their work.GitHub is establishing a $1 million legal defense fund for developers.Sidebar: EFF has just launched How to Fix the Internet, a new podcast mini-series that examines potential solutions to six ills facing the modern digital landscape.

    Brian #4: Install & Configure MongoDB on the Raspberry Pi

    Mark SmithDefinitely a “wow, I didn’t know you could do that” article.Tutorial walks throughInstalling 64 bit Ubuntu Server on a Raspberry PiConfigure wifiInstall MongoDB on PiSet up a user account, to safely expose MongoDB on a home network.Now you’ve got a MongoDB server in your house. So cool

    Michael #5: Extra! extra! extra!, hear all about it!

    Follow up on my critique of things like SQL & CSS put next to Python and Java. Maybe best to grab the conversation from here.Guido joins Microsoft, why? People seem to see this as a positive for sure. But they checked him out!New code editor roaming the streets: Nova from Panic.Two thumbs up on Big Sur and now waiting on the Mac Mini M1.

    Brian #6: A Python driven AI Stylist Inspired by Social Media

    Dale MarkowitzA bunch of Google tools (cloud storage, firebase, cloud vision api, product search api)Some React for front endPython to batch scriptGeneral oversimplified process:photos from social media for inspirationphotos of everything in your closet, multiple of each itemuse AI suggest outfits from your closet that match inspiration photosOk. The process is really more of a promo for Google AI products, and not so much about Python, but it’s a cool “look what you can do with software” kinda thing.Also, many of the tools used by online retail, like “similar products” and such, are available to lots of people now, and that’s cool.

    Joke:

    Back to the [dev] future!