## Fast XOR in Python

As I [have hinted at before](/2017/09/20/just-some-friendly-advice/), the [PyCrypto library](https://www.dlitz.net/software/pycrypto/) [seems to be dead](https://github.com/dlitz/pycrypto/issues/173). The [PyCryptodome](https://www.pycryptodome.org/en/latest/) library is a fork that is promising because it is maintained and works in Python 3, but they have a bit of a finger-wagging attitude which sometimes means that you have to fight the library a bit:
>>> from Crypto.Cipher import ARC4
>>> cipher = ARC4.new(B'funk')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python37\lib\site-packages\Crypto\Cipher\ARC4.py", line 132, in new
return ARC4Cipher(key, *args, **kwargs)
File "C:\Python37\lib\site-packages\Crypto\Cipher\ARC4.py", line 57, in __init__
len(key))
ValueError: Incorrect ARC4 key length (4 bytes)
>>> ARC4.key_size = range(1,257)
>>> ARC4.new(B'funk').decrypt( ARC4.new(B'funk').encrypt( B'Hello World' ))
b'Hello World'

They certainly mean well, but the library is no place to impose security standards, in my opinion. In malware research for example, we often have to verbatim copy the appalling use of certain ciphers, like ARC4 with a 4-byte key. It happens all the time! I have been particularly struggling with [the removal of the XOR cipher](https://pycryptodome.readthedocs.io/en/latest/src/vs_pycrypto.html). The XOR implementation of PyCrypto was very fast, and in this article I will both benchmark how fast exactly it was and give you a drop-in replacement which degrades gracefully based on your options. Do you want to know more?

## Get the MSDN Library for offline use

So you develop in [Microsoft Visual Studio Community Edition](https://www.visualstudio.com/de/vs/community/) and you long for the old days when there was a way to get the [MSDN Library](https://msdn.microsoft.com/en-us/library/) as an offline help file? Fear not, you still can. Open Visual Studio, type Ctrl+Q to open the quick access bar, usually located in the upper right corner of your interface. Enter Help Viewer, it should yield one result by that name, marked as an *"individual component"*. Selecting that entry should allow you to download and install the Help Viewer. Now relaunch Visual Studio and start the Help Viewer via quick access in the same way. You will be prompted whether you want to download some *content* - and I bet you do.

## Python: post JSON data containing a datetime object with requests to flask

Spoiler: My main point in this post is not given away by the title. But first things first: What are all those words? Would you like to know more?

Part of me wants to write about all the [horror](https://www.bleepingcomputer.com/news/security/ten-malicious-libraries-found-on-pypi-python-package-index/) and [glory](https://crates.io) there is to be seen in package management, but quite frankly it'll take too long. Instead, I will just leave you with a tiny piece of advice. Here comes. If you are on Windows and you want to install a *legitimate* Python package (like for example [PyCryptodome](https://pypi.python.org/pypi/pycryptodome), because naturally you are **fully aware** that [PyCrypto is dead](https://github.com/dlitz/pycrypto/issues/173).), which in reality is a bottomless pit, at the center of which there is a C library, straight from hell - then maybe get the [Microsoft Compiler for Python](http://www.microsoft.com/en-us/download/details.aspx?id=44266) instead of, who knows, wasting hours or even days looking for a less reasonable solution. Credit, as so very often, [goes to stackoverflow](https://stackoverflow.com/a/27327236/1578458).

## GithubTodos

1. https://github.com/dear-github/dear-github []
2. https://github.com/isaacs/github/issues []
3. on a side note, tellmewhenitcloses.com is pretty handy to avoid too many notifications in the first place []
5. Using local storage is handy for people that are not very concerned about privacy and just use the cloud synchronization feature of their browser: the content of their ToDo list will then also just be synced to all their devices. []

## Denied Permissions: copying text

Obviously, my Bank does not provide a REST API to download the transactions happening on my accounts. After I asked for "machine parseable" data, they told me that I can download CSV files. Awesome! So I wrote a parser and lived happily ever after. Except that they change their CSV format without notice every few months and at some point they started to mix different encodings in the same file. So I lived unhapply, regularly fixing the script reading the CSV file. What did not change for around 10 years now are their banking statements. And this also holds for the PDFs you have to download, if you want to avoid getting them via snail mail (and paying for the postage of course). I decided to parse the PDFs instead and this went pretty well for may years... up to recently, when something changed (it may have been a software update of the parser I use or something on their side). Do you want to know more?

## Code Rust in Windows

I have started to [learn](http://rustbyexample.com/) [rust](http://rust-lang.org), and I am enjoying myself. This is a merge and update on the two fantastic blog posts on [how to setup Visual Studio Code for Rust](https://mobiarch.wordpress.com/2015/06/16/rust-using-visual-studio-code/) and [how to enable debugging](https://sherryummen.in/2016/09/02/debugging-rust-on-windows-using-visual-studio-code/). In my personal opinion, from among [all the available IDE solutions for rust](https://areweideyet.com/), this is the best. Do you want to know more?

So unless you know and use [git annex](https://git-annex.branchable.com/), this is not going to be very useful for you. Check it out, though. It's pretty cool. Unless you are on Windows. In that case it's hell. Anyway, I wrote a script to help me figure out the output of [git annex unused](http://manpages.ubuntu.com/manpages/wily/man1/git-annex-unused.1.html). In short, it tells you what those files used to be called before you lost them. Script:
#!/usr/local/bin/python3.5
import re, sys, os
from subprocess import Popen,PIPE
FNULL = open(os.devnull, 'w')

def seeker(s):
process = Popen(["git", "log", "--stat", "-S", s], stderr=FNULL, stdout=PIPE)
match = re.search(r"(([ -~]*\/)*[ -~]*)\|", log)
if not match: return ''
else: return match.group(1).strip()

if len(sys.argv)>1:
print( seeker(sys.argv[1]) )
else:
clist = []
while True:
crawl = input().strip()
if not crawl: break
crawl = crawl.split()
clist.append(crawl)
for x in clist:
print('%s : %s' % ( x[0], seeker(x[1]) ) )

You can either call it with one argument which should be a key, or if you call it with no argument, it expects you to paste the list of results you got from git annex unused into stdin. It then goes through the list and tells you the corresponding filename for each key. This script is phenomenally stupid in that it does quite a terrible regular expression search on the output of git log and returns the first match it finds. Sue me, it works pretty well at my end.

## Binary Determinantal Complexity

My good friend and colleague Christian Ikenmeyer and I wrote this cute preprint about polynomials and how they can be written as the determinant of a matrix with entries equal to zero, one and indeterminantes. Go ahead and read it if you know even just a little math, it's quite straightforward. The algorithm described in section 3 has been implemented and you can download the code from my website at the TU Berlin. Compilation instructions are in ptest.c, but you will need to get nauty to perform the entire computerized proof.

## Switching the switching of Keys!

I recently lamented about switching two keys on my new Lenovo Yoga. Big problem: In my office, I attach that notebook to a docking station and to that docking station I attach a keyboard. On that keyboard, all keys are precisely the way I want them to be. Therefore, I do not want to switch the Insert and End keys when I am docked. I ended up writing a little batch script based on this nice google code wiki entry for the registry update and this stackexchange answer to elevate the batch script:
@ECHO OFF
NET FILE 1>NUL 2>NUL
if '%ERRORLEVEL%' == '0' goto run
powershell "saps -filepath %0 -verb runas" >nul 2>&1
goto eof
:run
REG QUERY "HKLM\SYSTEM\CurrentControlSet\Control\Keyboard Layout" ^
/v "Scancode Map" >nul 2>&1
IF '%ERRORLEVEL%' == '0' goto remove
<nul set /p ="> adding scancode map "
/v "Scancode Map" /t REG_BINARY /f ^
/d 00000000000000000300000052E04FE04FE052E000000000 >nul 2>&1
IF '%ERRORLEVEL%' == '0' goto success
goto fail
:remove
<nul set /p ="> removing scancode map "
REG DELETE "HKLM\SYSTEM\CurrentControlSet\Control\Keyboard Layout" ^
/v "Scancode Map" /f >nul 2>&1
IF '%ERRORLEVEL%' == '0' goto success
goto fail
:fail
echo failed.
pause
goto eof
:success
echo succeeded.
pause

Sadly, it always requires a reboot for the changes to take effect.

## Bit Twiddling Hacks

I recently implemented an algorithm that has to perform checks on all subsets of some large set. A subset of an $n$-sized set can be understood as a binary string of length $n$ where bit $i$ is set if and only if the $i$-th element is in the subset. During my search for code to enumerate such bitstrings, I found the greatest page in the entire internet. If anyone can explain to me how computing the next bit permutation (the last version) works, please do.

## Simple WordPress Update Script

I need to update this wordpress install every once in a while. There are lots of bash scripts on the internet that perform this task, and they are complicated beyond reason. This is what I use:
function cfg {
grep $2$1/wp-config.php | awk 'BEGIN {FS="[, )\x27]*"}; {print $3;}' } echo "> backing up database." mysqldump --user=$(cfg $1 DB_USER) \ --password=$(cfg $1 DB_PASSWORD) \ --host=$(cfg $1 DB_HOST) \$(cfg $1 DB_NAME) > backup.database.sql echo "> backing up website." tar -cjf backup.files.bz2$1

echo "> retrieving latest wordpress."
wget -q https://wordpress.org/latest.zip
unzip -qq latest.zip

echo "> updating wordpress."
rm -r $1/wp-includes$1/wp-admin
cp -r wordpress/* \$1/

echo "> cleaning up."
rm -r wordpress
rm latest.zip

It takes a single argument, which is the name of your wordpress root directory. It backups your database to the file backup.database.sql and backups the files to backup.files.bz2, then it simply proceeds as described in the wordpress codex for updating manual. I do not see what all the fuzz is about.

## Member Variables in Python

Member variables in python are horrible. They are not visible in the layout of the class which is instantiated, but instead the __init__ function of a class creates certain member variables for the instance. I have never liked this about python, to be honest. For a recent project, I devised the following solution. Assume you would want this behaviour:
>>> class test(Base):
...     # Variables
...     number = 4
...     string = "hodor"
...     # Functions
...     def stringmult(self):
...         return self.number * self.string
...
>>> test().stringmult()
'hodorhodorhodorhodor'
>>> test(number=2).stringmult()
'hodorhodor'
>>> test(string="Na",number=8).stringmult() + " - Batman!"
'NaNaNaNaNaNaNaNa - Batman!'
>>>
>>> test(end="Batman!")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ctypes.ArgumentError
>>>

In other words, any class that inherits from Base can be constructed with keyword arguments who must match exactly the correct class variables which you specify. This one does it:
from ctypes import ArgumentError
class Base(object):
def __init__(self, **kwargs):
# "given" is the list of keyword arguments passed to the constructor
# of this object, "needed" is the list of class variables which belong to
# the base class of the object which is being created, which do not end
# with two underscores and which are not a function. Trust me, we do not
# want to meddle with those.
given = list(kwargs.keys())
needed = [attr for attr in dir(self.__class__) if attr[-2:] != '__' \
and type(self.__class__.__dict__[attr])!=type(lambda:0) ]

# Check if keyword arguments have been provided which are not among the
# required arguments and throw an exception if so. Remove this check for
# a less restrictive base class. I wouldn't recommend it.
if not set(given) <= set(needed):
raise ArgumentError()

# First, initialize the attribute dictionary of the object being created
# with a list of default values, indicated by the values of the class
# variables. Then, update the attribute dictionary again with the values
# provided to this constructor.
self.__dict__.update({k: self.__class__.__dict__[k] for k in needed})
self.__dict__.update(kwargs)

I personally like this approach a lot and hereby dare you to tell me even a single reason not to do this, in the comments.