First, I changed modifier keys on the Mac. The main difference is that Mac uses the `cmd`

key while Linux / Windows uses the `ctrl`

key.

This gets back most of the familiar commands: `ctrl+c`

, `ctrl+v`

, etc. I didn't change the function of the original `cmd`

key because I still needed my `alt tab`

equivalent

Then I created a file at `~/Library/KeyBindings/DefaultKeyBinding.dict`

. This is what is in my file (modified from Ben Ogle's blog):

```
/* ~/Library/KeyBindings/DefaultKeyBinding.Dict
This file remaps the key bindings of a single user on Mac OS X 10.5 to more closely
match default behavior on Windows systems. This particular mapping assumes
that you have also switched the Control and Command keys already.
This key mapping is more appropriate after switching Ctrl for Command in this menu:
Apple->System Preferences->Keyboard & Mouse->Keyboard->Modifier Keys...->
Change Control Key to Command
Change Command key to Control
This applies to OS X 10.5 and possibly other versions.
Here is a rough cheatsheet for syntax.
Key Modifiers
^ : Ctrl
$ : Shift
~ : Option (Alt)
@ : Command (Apple)
# : Numeric Keypad
Non-Printable Key Codes
Up Arrow: \UF700 Backspace: \U0008 F1: \UF704
Down Arrow: \UF701 Tab: \U0009 F2: \UF705
Left Arrow: \UF702 Escape: \U001B F3: \UF706
Right Arrow: \UF703 Enter: \U000A ...
Insert: \UF727 Page Up: \UF72C
Delete: \UF728 Page Down: \UF72D
Home: \UF729 Print Screen: \UF72E
End: \UF72B Scroll Lock: \UF72F
Break: \UF732 Pause: \UF730
SysReq: \UF731 Menu: \UF735
Help: \UF746
NOTE: typically the Windows 'Insert' key is mapped to what Macs call 'Help'.
Regular Mac keyboards don't even have the Insert key, but provide 'Fn' instead,
which is completely different.
*/
{
"\UF729" = "moveToBeginningOfLine:"; /* Home */
"@\UF729" = "moveToBeginningOfDocument:"; /* Cmd + Home */
"$\UF729" = "moveToBeginningOfLineAndModifySelection:"; /* Shift + Home */
"$@\UF729" = "moveToBeginningOfDocumentAndModifySelection:"; /* Shift + Cmd + Home */
"\UF72B" = "moveToEndOfLine:"; /* End */
"@\UF72B" = "moveToEndOfDocument:"; /* Cmd + End */
"$\UF72B" = "moveToEndOfLineAndModifySelection:"; /* Shift + End */
"$@\UF72B" = "moveToEndOfDocumentAndModifySelection:"; /* Shift + Cmd + End */
"^\UF702" = "moveToBeginningOfLine:"; /* Home */
"^\UF703" = "moveToEndOfLine:"; /* End */
"$^\UF703" = "moveToEndOfLineAndModifySelection:"; /* Shift + End */
"$^\UF702" = "moveToBeginningOfLineAndModifySelection:"; /* Shift + Home */
"\UF72C" = "pageUp:"; /* PageUp */
"\UF72D" = "pageDown:"; /* PageDown */
"@x" = "cut:"; /* Shift + Del */
"@v" = "paste:"; /* Shift + Help */
"@c" = "copy:"; /* Cmd + Help (Ins) */
"@\UF702" = "moveWordBackward:"; /* Cmd + LeftArrow */
"@\UF703" = "moveWordForward:"; /* Cmd + RightArrow */
"$@\UF702" = "moveWordBackwardAndModifySelection:"; /* Shift + Cmd + Leftarrow */
"$@\UF703" = "moveWordForwardAndModifySelection:"; /* Shift + Cmd + Rightarrow */
"@\U007f" = ( deleteWordBackward:);
}
```

This only works if you have changed the modifier keys mapping. And voila, it sort of works now and other than some IDE shortcut keymap customisation it hasn't been much of a hassle switching between the two computers.

]]>If you want to play along, ask yourself this question: if a certain equation of motion satisfies the second law, does it necessarily satisfy the first? My first answer is yes, but after recent thought I realized it might not be.

I recently had a conversation with a friend about this, which started because I came across an interesting artifact called Norton's Dome, a thought experiment proposed by John Norton that seems to violate determinism even in Newtonian physics. Norton gives the dome a longer exposition on his website, but the basic idea is that there is a dome described by the equation

$$h(r) = \frac{2}{3g} r^{3/2}$$

on which the a ball is resting, at the very top. At this point, it is at equilibrium (though unstable). Any perturbation will push it rolling down the dome, but we assume there is none.

The simplest solution is that the ball remains forever at rest. However, due to the mathematical properties of the dome, a second solution is admitted, and hence one can write the solution as

$r(t) = (1/144) (t-T)^4 \text{ for } t \geq T$

$r(t) = 0 \text{ for } t < T$.

which means that at some arbitrary time $T$, the ball will start rolling for no reason at all. With all the above information, it is impossible to determine time $T$ at which it starts rolling.

Ok, so it seems there is this seemingly nonsense mathematical solution, which is fine. It is not uncommon in physics to reject solutions that are mathematically correct and physically wrong.

But is it physically wrong?

The first reaction is probably that there seems to be no cause to this sudden motion. But that's not a problem. Newton's laws only describe motion, they say nothing about cause and effect.

It agrees with Newton's second law. It also agrees with time reversal symmetry. One can imagine the time-reversed situation where the ball is propelled from bottom to top, with just the right energy to stop at the top (this is only possible for this specific dome shape, others will require an infinite time for this to happen). The ascent could happen at any time, so if you reverse time, it could also set off at any time in the future.

Norton also claims that it agrees with the first law, and this is where it gets a bit iffy.

The first law says:

Every object will remain at rest or in uniform motion in a straight line unless compelled to change its state by the action of an external force.

Norton interprets the first law as such: that in the absence of a net external force, a body is unaccelerated.

But by only applying the law at an instant, I think he is discarding a lot of what the word "remain" means. "Remain" necessary implies a time period, and by a normal interpretation of the word, I am still not convinced Norton's solution obeys the first law. The ball is at rest with no net external force acting on it, and clearly first law states it should *remain* at rest; rolling down the hill at some arbitrary time is a direct violation of that fact.

Which is the more right interpretation? You decide. I would say I am not quite convinced by Norton's.

And here we have a situation where the equation of motion satisfies the second law but, based on my interpretation, does not satisfy the first. That must necessarily mean that the first law says something that the second law doesn't, what is it?

In fact, if we look closely at the solution, we might notice where it goes wrong. At time $T$, acceleration is still zero, but the 4th order derivative, called snap / jounce, is not. This means at some arbitrary time $T$ the snap of the ball was instantaneously bumped up even though nobody touched the ball, and this "caused" the ball to begin its descent.

Norton thinks this is fine, but I do think this is forbidden by Newtonian mechanics. The implication in Newton's first law must be that when there is no external force, all higher order derivatives must necessarily be zero, for the ball to *remain* at rest.

So I think this is the important thing that Newton's first law says that the second law doesn't: while the second law talks about the second derivative of position, the first law imposes a specific condition on all higher derivatives.

But what happens then to time reversal symmetry? I have no answer to that. What do you think? Either way, though, I doubt the ball is going anywhere.

This is probably long overdue, but I really want to write about my masters project while it is still fresh on my mind (masters thesis, dissertation, or final year project report, whatever you want to call it. I would like to call it a thesis since it sounds loads cooler than the rest, but unfortunately my school just calls it the *project report*). It was submitted some time ago, and it is something I am quite proud of.

It didn't really start on a good note: the project was supposed to be done in pairs (though all assessments are separate) but my partner couldn't come back for fourth year due to personal reasons, and I had to do it alone. I sent a frantic email to my supervisor, and he told me that there is no problem doing this project alone. Though, it was also him who said that to get started with this project, "you just need to learn a *little* bit of General Relativity, shouldn't be too difficult."

He also mentioned that two other people (a PhD student and a postdoc) have previously attempted this project, but both ran out of time before they could finish it. So I was not particularly optimistic.

But it did end well. Here are the excerpts of the continuous assessment comments from my supervisor:

Lingyi has done an outstanding job with this project, showing great initiative in researching the literature very well, finding more than one relevant paper that I was not aware of. The results may be publishable in a journal paper, dependent on a period of digestion and the acquisition of understanding of what they mean, since there are a number of subtleties to comprehend. She has shown a great deal of independence, critical thinking, and self-motivation, looking at new ways of tackling the problem, and requiring only a small amount of guidance, and has worked hard.

and some comments on the viva:

Lingyi has read all of the literature that I am aware of, and some I was not, but here in the talk she concentrated on a few key papers, and she chose exactly the right ones.

Lingyi answered the questions with complete authority. She has clearly mastered her topic, and all the answers were correct.

I'm quite pleased, to say the least.

So, what was it about? There have been some previous posts that spun off my final year project (see Distances in Schwarzschild lensing and Numerical integration of light paths in a Schwarzschild metric) that might give you an idea. The one line summary is this: we want to know if the cosmological constant affects gravitational lensing.

This question basically splits the research into two camps: one believes yes, the other believes no. My supervisor is in the "no" camp.

Of course this statement is not entirely accurate and it is more complicated than that, such as the fact that gravitational lensing in a cosmological context depends on angular diameter distance which already has a dependence on the cosmological constant, so the real question is whether the current formula is enough, and also that angle and distance measurements are not always clearly defined when space is not asymptotically flat, so we have to clearly state what we refer to when we say "bending angle". But regardless, this is the essential question. Surprisingly, given how widely used gravitational lensing is and how widely accepted the cosmological constant is, there is not yet a consensus on this. We hope to answer the question numerically instead of analytically.

Anyway, this post was not meant to be a full description of what my project is—I have my report for that. If you're curious, our answer to whether the cosmological constant affects gravitational lensing is, as with most research questions: maybe. More specifically, due to the intricacies of our cosmological model, it is difficult to truly isolate the effect of the cosmological constant on lensing.

I briefly considered putting a graph of my results here, but I think it is rather irresponsible to post a results graph without explaining its intricacies and nuanced implications, and it is difficult to do that in a single blog post without an exposition of the background literature.

There is no doubt that I loved the journey, though there were multiple stressul points along the way. It has always been my dream to do astronomy-related research and I can't thank my supervisor Alan Heavens enough for proposing this project that is at the same time both challenging and not undoable. The project answers a real research question and the process of doing it makes me feel like I am genuinely contributing to the pool of collective scientific knowledge.

Of couse, self-doubt was frequent and imposter syndrome plagued me along the way...

But I did it! It was an amazing feeling that nearer the second half of my project, I could understand most of the research papers that I struggled with initially, and more than that, I also understood some of their problems. It's sad that perhaps this is my last contact with true physics research for a long time (or likely for good), and this project was important enough in my life that it deserves a formal goodbye.

]]>I feel like there is always a cycle. I find something new, fiddle with it, keep adding cool new functionality to it, and eventually I get tired of the bloat and revert to the minimalistic solution.

And maybe someday I will get bored of this simple design and start adding things to it again. So if the version you see has any of those things listed above, you'd know that the cycle has begun once more.

]]>In the project I have been dealing a lot with gravitational lensing by a Schwarzschild black hole. The famous formula for the bending angle (in units of $c = G = 1$) is

$$\alpha = \frac{4M}{R}.$$

This formula forms the basis for one of the key observational confirmations of Einstein's theory of General Relativity. The Newtonian prediction is only half of the GR result.

In the formula, $M$ is the lensing mass. In the Schwarzschild spacetime, this is the mass of the black hole. But when I first started out on my project, I was a little confused about which distance $R$ was referring to.

There are 3 distances that are typically used as length measures:

- $b$, the impact parameter, defined as the perpendicular distance between the light path and the black hole
- $r_0$, the distance of closest approach between the light path and the black hole
- $R$, the perpendicular distance between the unperturbed trajectory and the black hole

Their differences between them can be seen in the diagram below.

Most papers use the impact parameter $b$ to parameterize lensing, because it is a physical, measurable quantity (though complications also arise when you throw in a cosmological constant, see Lebedev & Lake, 2013), whereas the distance of closest approach is not so easy to determine, and $R$ is a result of a mathematical construct which (from what I understand), roughly corresponds to the distance at turning point from the black hole assuming the light ray was bent at a single point.

These distances are similar, but not the same. They are all equal to one another to first order. This means that to first order in $M/R$, the bending angle can be written equivalently as

$$\alpha = \frac{4M}{R} = \frac{4M}{r_0} = \frac{4M}{b}.$$

However, they can no longer be used interchangeably from second order onwards. It is easy to convert between them using the relationships between the three distance measures:

$$\frac{r_0}{b} = 1 - \frac{M}{b} - \frac{3}{2}\left (\frac{M}{b} \right )^2 - 4\left (\frac{M}{b}\right )^3 + ...$$

$$\frac{R}{r_0} = 1 + \frac{M}{R} + \frac{3}{16} \left (\frac{M}{R} \right )^2 + ...$$

For example, here is a table of the second and third order coefficients for the different distance measures:

2nd order | 3rd order | |
---|---|---|

$M/R$ | $15\pi/4$ | $401/12$ |

$M/r_0$ | $-4 + 15\pi/4 $ | $122/3 - 15\pi/2$ |

$M/b$ | $15\pi/4$ | $128/3$ |

If even higher orders are needed, Keeton and Petters (2005) contains a detailed derivation of the lensing formula up to arbitrary order. Most of the time, though, just the first order term will suffice. But in my project I am investigating much smaller effects in gravitational lensing, and the higher order terms are important.

]]>To find the trajectory of anything in General Relativity, usually you only need the metric tensor, from which you can obtain the geodesic equations. Nevertheless, a common problem that arises in cosmology is that as soon as we depart from the simplest homogeneous models, the task of finding solutions to the geodesic equations quickly becomes an intractable analytical problem.

In this post are some notes of how to perform numerical integration of light paths in the Schwarzschild metric.

The Schwarzschild metric is one of the most famous solutions to the Einstein field equations, and the line element in this metric (in natural units $c = G = 1$) is given by:

$$ds^2 = -f(r) dt^2 + \frac{dr^2}{f(r)} + r^2(d\theta^2 + \sin^2 \theta d\phi^2)$$

We are interested in the trajectory of a light ray in such a metric. Since the metric is spherically symmetric, any light ray that starts with a certain $\theta$ must stay in the same $\theta$ plane, hence we can arbitrarily set $\theta = \pi/2$ and do away with all the $\theta$ terms.

Light follows a null (lightlike) trajectory given by $ds^2 = 0$. In the absence of external forces, it should also travel along a geodesic. These are governed by the geodesic equations, which can be derived using Euler-Lagrange equations. Due to the symmetry of the metric, applying the Euler-Lagrange equations to the metric gives us two conserved quantities:

$$f \dot{t} = E = \text{constant}$$

$$r^2 \dot{\phi} = L = \text{constant}$$

where an overdot refers to derivative with respect to an affine parameter $\lambda$.

Using the null condition, we have

$$f\dot{t}^2 - \frac{\dot{r}^2}{f} - r^2\dot{\phi}^2 = 0$$

This can be expressed in terms of $\dot{r}$, and differentiating again gives the second-order differential equation for $r$:

$$\ddot{r} = \frac{L^2(r - 3M)}{r^4}$$

This can be easily converted into a first-order differential equation to be solved numerically by setting a variable $p = \dot{r}$. So we have these 3 differential equations to compute numerically:

$$\dot{r} = p$$

$$\dot{p} = \frac{L^2(r - 3M)}{r^4}$$

$$\dot{\phi} = \frac{L}{r^2}$$

(This can of course be solved analytically in the weak gravity limit, which gives the light bending equation $\Delta \phi = 4M/b$ where $b$ is the impact parameter.)

In principle, we need the initial values of $r$, $p$, and $\phi$ to start the numerical simulation. However, if we fix the incoming velocity to be horizontal, then we would only need to specify the initial $x_0$ and $y_0$ coordinates.

We can take $\dot{t} = 1$ for convenience. The initial conditions can then be derived from the null condition and they are as follows:

$$r = \sqrt{x_0^2 + b^2}$$

$$\phi = \cos^{-1} \left ( \frac{x_0}{r} \right )$$

$$\dot{r} = p = \cos{\phi}$$

$$L = r^2 \dot{\phi} = r \sqrt{1 - \dot{r}^2}$$

Then, the only free parameters to specify $b$ and $x_0$, in addition to mass.

For a mass of $M = 1$ (corresponding to Schwarzschild black hole radius of 2), this is a plot of the trajectories with different impact parameters $b$:

And they do fit quite well with the theoretical deflection angle $4M/b$ for large impact parameters:

]]>It's no secret that Jekyll's built in related posts functionality doesn't return related posts--it only gives you the most recent posts. So I looked around for better solutions.

One popular solution is to calculate related posts based on a post's tags. However, this requires one to conscientiously tag every post accurately, which seems unlikely. I wanted a much lazier solution--to calculate how related two posts are only based on their content.

The aim is simple: to have some way of calculating similarity score between two posts, then we can rank and take the top 5 posts as the related posts.

The common way of measuring document similarity is by transforming the set of documents into a tf-idf matrix, then computing the cosine similarity to get the similarity score of each document with all other documents. Tf-idf stands for term frequency-inverse document frequency, and it is composed of two parts:

**term frequency**: proportional to how often a word appears in a document. This accords importance to words that appear frequently in a document.**inverse document frequency**: indicates how often a word appears in all documents. If it appears many times in all documents, then it is given less importance. These are the words that are going to appear frequently no matter what the post is truly about.

The tf-idf matrix is a matrix where each column represents a document and each row represents a word, and each cell represents the tf-idf value of the word within the document. Other people have explained this better than me.

But before doing this, some tokenization and stemming is needed, since I wouldn't want words like "taken" and "taking" to be identified as two separate words. I also want to remove stop words, which are commonly-used words like "the", "a", "of".

Optimally, I would use Ruby to do this since Jekyll is based on Ruby, but unfortunately my Ruby knowledge is zero and I decided to use Python for this. For people who just want to see code, the scripts can be found on my GitHub repo.

First I wrote a function that processes the markdown files containing the posts:

```
import frontmatter
import glob
def get_posts(folder='./_posts'):
result = {}
for filepath in glob.glob(folder + "/*"):
filename = filepath.split('\\')[-1]
slug = filename[11:-3]
post = frontmatter.load(filepath)
if "slug" in post.keys():
slug = post["slug"]
result[slug] = post.content
return result
```

This just gets all the files from the `_posts/`

folder, gets the post slug and content, and stores the information in a dictionary. I used the `python-frontmatter`

module to deal with the yaml frontmatter.

Then I use the `nltk`

and `scikit-learn`

libraries to do the text processing. `nltk`

has the capabilities to do all the natural language processing shenanigans I need while `scikit-learn`

has the `TfidVectorizer`

to turn documents into a tf-idf vector.

First, some standard stemming and tokenizing (where `nltk`

does most of the work):

```
import nltk
import string
from sklearn.feature_extraction.text import TfidfVectorizer
stemmer = nltk.stem.porter.PorterStemmer()
nltk.download('punkt') # download the needed data for tokenizing
def stem_tokens(tokens, stemmer):
stemmed = []
for item in tokens:
stemmed.append(stemmer.stem(item))
return stemmed
def tokenize(text):
tokens = nltk.word_tokenize(text)
stems = stem_tokens(tokens, stemmer)
return stems
```

Then I create the vectorizer with the `tokenize`

function, and clean the data by lowercasing everything and removing punctuation, and put the data into the vectorizer.

```
vectorizer = TfidfVectorizer(tokenizer=tokenize, stop_words='english')
posts = get_posts()
# lowercase and remove punctuation from post data
cleaned_posts = {slug: post.lower().translate(str.maketrans('', '', string.punctuation)) for slug, post in posts.items()}
slugs = list(cleaned_posts.keys())
tfidf = vectorizer.fit_transform(list(cleaned_posts.values()))
matrix = (tfidf * tfidf.T).A # calculate cosine similarity
# example matrix:
# [[ 1. 0.12274921 0.08471414 0.0465803 0.04871383 0.00808005
# 0.0196523 ]
# [ 0.12274921 1. 0.10744334 0.20886152 0.07531169 0.0452097
# 0.04654832]
# [ 0.08471414 0.10744334 1. 0.05036088 0.0453141 0.02618316
# 0.04787127]
# [ 0.0465803 0.20886152 0.05036088 1. 0.16894053 0.03408972
# 0.03633891]
# [ 0.04871383 0.07531169 0.0453141 0.16894053 1. 0.03106121
# 0.03287819]
# [ 0.00808005 0.0452097 0.02618316 0.03408972 0.03106121 1.
# 0.02760873]
# [ 0.0196523 0.04654832 0.04787127 0.03633891 0.03287819 0.02760873
# 1. ]]
```

The last calculation returns a symmetric matrix $M$ where the $M_{ij}$ is the similarity between document $i$ and document $j$. As a check, we can see that the elements on the diagonal are unity since a document's similarity with itself should be 1.

Then we just need to sort through the matrix to get the top $n$ most related posts. In this case, I took $n = 3$.

```
num_best = 3
result = {}
for i, row in enumerate(matrix):
indices = row.argsort()[-num_best-1:-1][::-1]
current_slug = slugs[i]
result[current_slug] = [slugs[index] for index in indices]
# related posts are now stored in the result variable
```

Now, we have to find some way of inserting this result back into Jekyll. To do this, I decided to make use of data files to be accessed by my layouts:

```
def write_result_to_file(related, file='./_data/related.yml'):
data = []
for r in related:
r = {
'post': r,
'related': related[r]
}
data.append(r)
with open(file, 'w') as f:
yaml.dump(data, f, default_flow_style=False)
# then we just have to put the previously calculated result into this function
write_result_to_file(result)
```

This function does some processing with the dictionary it receives and dumps it into `_data/related.yml`

. The resulting file should look something like this:

```
- post: setting-up-jekyll
related:
- drifter-writing-interactive-fiction-with-ink
- getting-better-related-posts-in-jekyll-using-tf-idf
- git-for-noobs
- post: drifter-writing-interactive-fiction-with-ink
related:
- solving-the-24-game
- setting-up-jekyll
- git-for-noobs
```

Inserting this data into my `post`

layout was more difficult than I thought, because I could find no way of getting the post object from the post slug. So, I ended up with this ugly code in my `_layouts/post.html`

:

```
{% for item in site.data.related %}
{% if page.slug == item.post %}
<nav class="read-next">
<h3 class="read-next-label">Other posts you might enjoy</h3>
<ul>
{% for pslug in item.related %}
{% for p in site.posts %}
{% if p.slug == pslug %}
<li><a class="read-next-title" href="{{ p.url | prepend: site.baseurl }}" title="{{ p.title | xml_escape }}">{{ p.title | xml_escape }}</a></li>
{% endif %}
{% endfor %}
{% endfor %}
</ul>
</nav>
{% endif %}
{% endfor %}
```

Ew, 3 nested for loops. But I couldn't find a way out of it--the first loop is to find the relevant element in the list that the post corresponds to, by checking the slugs. The second to loop through the slugs, so that I can render them, however, to get the post object from the slug, the third loop is needed to loop through all the posts to see which post has a slug that matches.

Fortunately, this is only run when building the site, and doesn't slow down things on the client side. I only have a handful of posts on my blog, so it really makes no difference. The inconvenience it may bring to some people is that you would have to do the extra step of running the python file to generate `_data/related.yml`

before building the site. But I use gulp to build my site, so I just had to add an extra line in my gulpfile:

```
shell.exec('python scripts/similarity.py')
```

This is roughly similar to this article on Google Cloud, just a stripped and slightly reorganized version.

To enable HTTPS, we need certificates. So first we need to create a directory to hold the certs.

```
cd
mkdir certs
```

We use the Docker Let's Encrypt nginx-proxy companion to automatically issue and use signed certificates. To do this, we declare volumes when running the reverse-proxy so the `nginx-letsencrypt`

companion can populate them with certificates.

However, in order for `nginx-proxy`

to proxy other containers, they have to be on the same Docker network.

So we first create a network:

`docker network create --driver bridge reverse-proxy`

And we run the nginx reverse proxy and the letsencrypt companion on this network, using the `--net reverse-proxy`

command.

Running the nginx reverse proxy:

`docker run -d -p 80:80 -p 443:443 --name nginx-proxy --net reverse-proxy -v $HOME/certs:/etc/nginx/certs:ro -v /etc/nginx/vhost.d -v /usr/share/nginx/html -v /var/run/docker.sock:/tmp/docker.sock:ro --label com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy=true jwilder/nginx-proxy`

Running the `nginx-letsencrypt`

companion:

`docker run -d --name nginx-letsencrypt --net reverse-proxy --volumes-from nginx-proxy -v $HOME/certs:/etc/nginx/certs:rw -v /var/run/docker.sock:/var/run/docker.sock:ro jrcs/letsencrypt-nginx-proxy-companion`

We can then start all our website or other app containers using a `docker-compose.yml`

file. For example, my docker-compose file looks something like this:

```
version: '2'
services:
blog:
restart: always
image: nginx
container_name: blog
volumes:
- "/etc/nginx/nginx.conf:/etc/nginx/nginx.conf"
- "/var/www/blog:/etc/nginx/html"
networks:
- reverse-proxy
environment:
- VIRTUAL_PORT=1234
- VIRTUAL_HOST=theconfused.me
- LETSENCRYPT_HOST=theconfused.me
- LETSENCRYPT_EMAIL=lingyihuu@gmail.com
draggymail:
restart: always
build: "/var/www/draggymail"
container_name: draggymail
networks:
- reverse-proxy
environment:
- VIRTUAL_PORT=1234
- VIRTUAL_HOST=draggymail.theconfused.me
- LETSENCRYPT_HOST=draggymail.theconfused.me
- LETSENCRYPT_EMAIL=lingyihuu@gmail.com
networks:
reverse-proxy:
external:
name: reverse-proxy
```

The blog service serves up my blog, which is what you're looking at right now. It just uses the `nginx`

container. You can insert the nginx configuration by mounting the appropriate configuration file. The format for mounting volumes is

`/path/on/host:/path/in/docker/container:options`

The `options`

field is optional, it is where you can specify what docker is allowed to do with the mounted volume. `ro`

stands for readonly, and `rw`

gives the docker container permission to (yup, you guessed it) read and write.

The draggymail service builds and runs my NodeJS app. The filepath in `build`

tells docker to look for the `Dockerfile`

inside the folder, build the image, and run the container. This is so that I don't have to build the image separately every time.

Then just run `docker-compose up -d`

to run your containers.

A few docker commands I found myself repeatedly searching up.

`docker build -t image_name:tag_name .`

Simple command:

`docker run image_name:tag_name`

A more complicated command:

`docker run -d --name container_name -p 80:80 -p 443:443 --net network_name -v /path/of/file/in/host:/path/of/file/in/container image_name:tag_name`

`-d`

detached mode`--name`

specify the name you want to call this container`-p`

ports mapping`--net`

network to put it on`-v`

volumes to mount

`docker network create --driver bridge network_name`

`--driver`

driver to manage the network

(thanks to this stackoverflow answer)

`docker start -a -i `docker ps -q -l` `

`docker start`

start a container (requires name or ID)`-a`

attach to container`-i`

interactive mode`docker ps`

List containers`-q`

list only container IDs`-l`

list only last created container

`docker exec -it container_name_or_id /bin/bash`

`docker rmi $(docker images -qa -f 'dangling=true')`

List all containers (running and exited):

`docker ps -a`

`dosomething`

whose argument is an instance of class A. That class A has an attribute which is an instance of class B, and B has an attribute containing a list of objects of class C. So the function looks something like this:
```
def dosomething(objA):
alist = objA.objB.some_attribute
for objC in alist:
print objC.name
print objC.value
```

To test this function, we would have to mock class A. Without MagicMock, we would need to create a mock for multiple classes, like so:

```
class C:
def __init__(self, name=None, value=None):
self.name = name
self.value = value
class B:
def __init__(self):
self.some_attribute = [] # this will be a list of instances of class C
class A:
def __init__(self):
self.objB = B()
```

Then to use this in your unit test:

```
import unittest
class RandomTests(unittest.TestCase):
def test_dosomething(self):
a = A()
a.objB.some_attribute = [C('somename', 'somevalue')]
dosomething(a)
# then assert one output
```

All these just to pass the correction argument into the function `dosomething`

so that it doesn't complain about attribute errors.

But with magic mock, we don't need to manually mock all our classes, and it just becomes:

```
import unittest
from mock import MagicMock
class RandomTests(unittest.TestCase):
def test_dosomething(self):
a = MagicMock()
a.objB.some_attribute = [MagicMock(name='somename', value='somevalue')]
dosomething(a)
# then assert on output
```

It's just like magic!

]]>From the Gmail API documentation, when you send a get request to the Gmail API for a particular `userId`

and `messageId`

, it returns a Users.message resource which looks something like this:

```
{
"id": string,
"threadId": string,
"labelIds": [
string
],
"snippet": string,
"historyId": unsigned long,
"internalDate": long,
"payload": {
"partId": string,
"mimeType": string,
"filename": string,
"headers": [
{
"name": string,
"value": string
}
],
"body": users.messages.attachments Resource,
"parts": [
(MessagePart)
]
},
"sizeEstimate": integer,
"raw": bytes
}
```

The recipients can be obtained from the `headers`

field, by taking the value corresponding to "to" in the name. Usually, with multiple recipients, the value looks something like this:

`"John Miranda" <john@gmail.com>, "Doe, Emily" <emilydoe@gmail.com>, something@example.com`

The recipients are comma separated and for each recipient there are 2 main formats it can come in:

- "Name" <email address>
- email address

There are a few things that has to be taken into account that makes the problem not so simple:

- if name is present, email address is enclosed in angular brackets
- otherwise, email address appears alone, without the angular brackets
- name is enclosed by quotation marks if there is a comma in the name

The end goal is to extract the recipients information from this response into an array with each item of the array being a dictionary containing the name and email of the sender, if they exist. It is obvious that we cannot simply split the string by commas, since there might also be a comma in the name.

The following javascript regex takes the above considerations into account and, as far as I've tested it, splits the string properly:

```
function parseEmailRecipients(recipientString) {
// an example string:
// const str = `"John Miranda" <john@gmail.com>, "Doe, Emily" <emilydoe@gmail.com>, something@example.com, John Doe <something@gmail.com>, `;
const regex = /(([\w,\"\s]+)\s)?<?([^@<\s]+@[^@\s>]+)>?,/g;
let m;
let recipientsArray = [];
while ((m = regex.exec(recipientString)) !== null) {
// This is necessary to avoid infinite loops with zero-width matches
if (m.index === regex.lastIndex) {
regex.lastIndex++;
}
let name = null;
let email = null;
if (m[2]) {
name = m[2].replace(/,$/, '').replace(/"/g, "").trim(); // strip whitespaces and commas, and remove quotation marks
};
if (m[3]) {
email = m[3].replace(/,$/, '').trim(); // strip whitespaces and commas from end of string
}
let item = {
name: name,
email: email,
};
recipientsArray.push(item)
}
return recipientsArray;
}
```

The main part of the function is in this regex `(([\w,\"\s]+)\s)?<?([^@<\s]+@[^@\s>]+)>?,`

, which can be broken down into three main parts:

`(([\w,\"\s]+)\s)?`

`<?([^@<\s]+@[^@\s>]+)>?`

`,`

The **first part** `(([\w,\"\s]+)\s)?`

optionally identifies the name. The outermost round brackets denote a group. Inside it, we have a square bracket that tells it to look for any of these characters in the list: `\w`

(any word character, that is, any letter, digit, or understore, equivalent to `[a-zA-Z0-9_]`

), `,`

, `"`

, and `\s`

(a whitespace character). The `+`

tells regex to match as many of these as possible, that is, it can match any character that belongs in the list any number of times. This is followed by a space again. The question mark that follows tells it that this whole group is optional - there may be some items without names appearing in front.

The **second part** `<?([^@<\s]+@[^@\s>]+)>?`

identifies the email. The opening and closing angle brackets are optional, therefore they are both followed by a question mark. The expression enclosed in round brackets describes the email format, which basically says look for a bunch of adjacent characters that are not `@`

or `\s`

, followed by a `@`

symbol, then followed by another bunch of characters that are not `@`

or `\s`

. This is then enclosed by optional angle brackets.

The **last comma** specifies the separator. Note that I don't think the last entry ends with a trailing comma, and this which would cause the regex to skip the last recipient when parsing. However this problem can be easily circumvented by manually adding a comma to the end of the string before using the regex.

`regex.exec`

then returns the groups that were identifies through the round brackets. After that, all that's left to do is to trim some residual spaces and commas from the end of the string and remove the quotation marks that sometimes encloses a name.

For example, given a list of numbers `[4,8,3,6]`

, we can get 24 by doing `4*8*(6/3)`

or `4*3*(8-6)`

. What you cannot do is take, say, `4*6`

and discard the rest.

Sometimes, there are also 4-number combinations that can never make 24. One example is `[3,9,4,10]`

. What usually happens in a game when we encounter such combinations is that we would stare at the cards in silence for a long period of time, before at some point we both agree that this is an impossible combination. But this is incredibly arbitrary; we might very well pass over combinations that are actually possible but neither of us could see.

One very commonly missed one was `[1,5,5,5]`

. You can have a try if you want to play along, but don't spend too long.

So I wanted to write a program to check if there is a solution to a given a set of numbers, and print the solution if it exists. I've tried this a few years back when I first started coding, but somehow just couldn't cover all the edge cases. Armed with slightly more programming knowledge and experience, I decided to give it another go.

My first instinct was to construct all possible expressions that are possible with the four numbers and check whether any of them equal 24, or any other number you choose. It is easy to permute the numbers and operations, but the main difficulty lies in the placement of brackets.

For four numbers, the possible positions of brackets are still iterable, though just barely, and makes the code look very ugly. For example, there is a solution on Rosetta Code that uses this principle; it specifies all the possible bracket insertion indices in an arithmetic expression string using the following code snippet:

```
brackets = ( [()] + [(x,y)
for x in range(0, exprlen, 2)
for y in range(x+4, exprlen+2, 2)
if (x,y) != (0,exprlen+1)]
+ [(0, 3+1, 4+2, 7+3)] ) # double brackets case
```

The empty tuple is the case with no brackets, the list comprehension after that generates all possible combinations of open brackets (variable `x`

) and close brackets (variable `y`

), and the third list is the indices of the brackets in the double bracket case, as in `(5 + 3) * (1 + 2)`

.

The major limitation of this solution is that it only works for four numbers, where it is easy to reason about the bracket placements. To make it work for more than 4 numbers, one would have to take into account double and triple brackets, nested brackets, and many more. In retrospect it might be possible to algorithmically generate possible bracket placements, possibly with some kind of recursive algorithm, but at that point it just felt paralyzingly messy.

I realized that to make it work for more numbers than four, I would have to take a different approach. I am pretty good at the game, so I thought about the strategy I use when playing the game and realized my first approach was wholly mechanical and unnatural.

I don't think about bracket placement when I play the game, neither do I try to construct all possible expressions from the four numbers and evaluate them one-by-one to see if they give the desired answer. Instead, given four numbers, what I would do mentally is this: I would try to combine pairs of numbers and reduce them, until there are two numbers remaining, and then check if there is any way these two numbers evaluate to 24.

For example, if we started with `[1,2,3,4]`

, this is roughly the thought process that occurs:

`[1,2,3,4] --> [1,6,4] --> [1,24] --> 24 --> success!`

As such, it becomes apparent that this is a recursive solution: solving for n numbers is the same as choosing to combine 2 numbers and solving for the remaining n-1 numbers.

Once I got this realization, writing the algorithm was not difficult. For a given list of numbers, I iterate through all possible pairs that can be generated from the list. For each pair, I go through the possible values that can be produced from the pair by using the 4 operators. For each value, I generate a list with one less element than the previous, and call the function again.

This contains two nested for loops, and might blow up for a large input array. But practically the 24 game is usually played with 4 numbers, and the algorithm does reasonably well with a list of similar order of magnitude.

The resulting python code looks like this, with the main solver logic in `solve()`

:

```
import itertools
def solve(numbers, goal=24, expr=[]):
if expr == []:
expr = [str(n) for n in numbers]
if len(numbers) == 1:
if numbers[0] == goal:
return numbers[0]
else:
return False
if len(numbers) == 2:
answers, answer_exps = combinetwo(numbers[0], numbers[1])
for i,answer in enumerate(answers):
if answer == goal:
return convert_expr_to_string(expr[0], expr[1], answer_exps[i])
return False
pairs = set(itertools.combinations(numbers, 2))
for pair in pairs:
possible_values, possible_expr = combinetwo(*pair)
for counter, value in enumerate(possible_values):
expression = possible_expr[counter]
a_index = numbers.index(pair[0])
b_index = numbers.index(pair[1])
if a_index == b_index:
b_index = numbers.index(pair[1], a_index + 1);
expr_string = convert_expr_to_string(expr[a_index], expr[b_index], expression)
newlist = numbers[:]
newexpr = expr[:]
# replace the two numbers with the combined result
a_index = newlist.index(pair[0])
newlist.pop(a_index)
b_index = newlist.index(pair[1])
newlist.pop(b_index)
newlist.append(value)
# order matters
newexpr.pop(a_index)
newexpr.pop(b_index)
newexpr.append(expr_string)
result = solve(newlist, goal, newexpr)
if result:
return remove_redundant_brackets(result)
else:
continue
def convert_expr_to_string(a, b, expr):
temp = [a, b]
result = '(' + str(temp[expr[0]]) + ')' + str(expr[1]) + '(' + str(temp[expr[2]]) + ')'
return result
def combinetwo(a, b):
result = [a + b, a * b]
expr = [(0, '+', 1), (0, '*', 1)]
if b > a:
result.append(b-a)
expr.append((1, '-', 0))
else:
result.append(a-b)
expr.append((0, '-', 1))
if b != 0:
result.append(a / b)
expr.append((0, '/', 1))
if a != 0:
result.append(b / a)
expr.append((1, '/', 0))
return result, expr
def remove_redundant_brackets(expr):
stack = []
# indices to be deleted
indices = []
for i, ch in enumerate(expr):
if ch == '(':
stack.append(i)
if ch == ')':
last_bracket_index = stack.pop()
enclosed = expr[last_bracket_index + 1:i]
if enclosed.isdigit():
indices.append(i)
indices.append(last_bracket_index)
return "".join([char for idx, char in enumerate(expr) if idx not in indices])
```

An example usage looks like this:

```
>>> solve([1, 5, 5, 5], goal=24)
'5*(5-(1/5))'
```

Of course you can pass in any goal and a list of arbitrary length. Currently, it should return nothing if there is no solution.

I also converted the python code above to javascript, and made a page so that people can play around with the game.

]]>