Noah's Cool Doggo Bloggo

Creating a tool to help answer yes/no questions in Irish, a language without words for "yes" or "no"

Noah Ó Donnaile — Wed, 15 Aug 2018 16:59:34 GMT

I had the pleasure of spending a couple of months this summer interning at the ADAPT Centre in Dublin City University, working with the GaelTech team of Dr. Teresa Lynn and PhD candidates Abigail Walsh and Meghan Dowling, on resources and tools for Irish language technology.

Foireann GaelTeic ag an ócáid intéirneachta inniu san @AdaptCentre! Tá sárobair déanta ag Noah maidir leis a chuid taighde ar theicneolaíocht na #Gaeilge pic.twitter.com/DJA2WlvKqF
— Meghan Dowling (@ismisemeg) July 18, 2018

I had a lot of fun while I was there, and learned a lot about research and the postgraduate life. I will finish my undergraduate degree soon, and I have no doubt that this experience will be extremely useful when I choose what to do afterwards!

I got to work on a few very interesting things at ADAPT. Below is a photo Meghan took of the poster I procrastinated in making that sums up some of my projects.

Tá grianghraif agam de, ach seans nach bhfuil sé ró-shoiléir. Is le @iandioch é agus tá @aneihe ag obair ar an stuif seo chomh maith, beidh tuilleadh eolais acu! pic.twitter.com/Tjvymx2l1J
— Meghan Dowling (@ismisemeg) July 18, 2018

However, not mentioned on the poster is my final project, which was to do with programmatically answering yes/no questions in Irish. Below, I explain how "yes" and "no" work in the language, try to show why this causes an issue in machine translation, before briefly explaining how I tried to get a computer to approach the problem.

"Yes" and "No" in Irish

Irish is a Celtic language, spoken (spoiler alert) on the island of Ireland. It has 2 grammatical genders, 4-ish grammatical cases, VSO word order, and has no words for "yes" or "no".

The lack of words for "yes" and "no" is surprising to many people I mention it to. If you were to translate the answers to the question "Is she ok?" to Irish, it would look something like this:

English	is she ok?	yes	no
Irish	an bhfuil sí ceart go leor?	tá	níl
Literal translation	is she alright?	is	isn't

As you can see, instead of "yes" or "no", you reply with "is" or "isn't". The person answering the question takes the verb from the question and repeats it for the response.

Another example:

English	will you go to the disco tonight?	yes	no
Irish	an rachaidh tú chuig an dioscó anocht?	rachaidh	ní rachaidh
Literal translation	will you go to the disco tonight?	will go	won't go

Machine translation systems struggle with differences in languages like this one. Google Translate is probably the most popular machine translation system in use right now; here is its attempt at translating something along these lines:

English	Will you go to the disco tonight? Yes? No?
Google Translate's Irish	An dtéann tú ar an dioscó anocht? Tá? Níl?
Literal translation of the "Irish"	Do you go to the disco tonight? Is? Isn't?

It is clear to see that this is a bad translation. In Irish, in order to correctly translate a "yes" or "no", you need to reference the question you were asked. You take the main verb, maybe inflect it in some way, and respond with it. You can't just translate the word "yes", in isolation; you need to know what you're saying "yes" to. Google Translate doesn't seem to know how to do this.

Languages are weird

The above examples are complicated by how English uses auxiliary verbs "will" and "won't" for its future tense. Also, English word order changes from being SVO ("he is a dog.") in normal phrases to being VSO ("is he a dog?") in questions. English is a weird language.

See the following for how the structure of a more normal phrase, instead of a question, compares in Irish and English:

English	You will go to the disco tonight.
Irish	Rachaidh tú chuig an dioscó anocht.
Literal translation	["go" in future tense] you to the disco tonight.

Irish doesn't have an auxiliary verb here like English does, and it is consistently VSO in phrases like this.

English is indeed weird in switching word order and using auxiliary verbs, but of course, Irish is weird in its own ways too. For example, try and figure out how the following translation makes sense:

English	do you have a dog?	yes	no
Irish	an bhfuil madra agat?	tá	níl
Literal translation	is a dog at you?	is	isn't

We saw earlier in the "is she ok?" example that "an bhfuil?" means "is?" in Irish. "is ?" doesn't seem like it can translate as "do you have a dog?", but let's see.

"madra" is the Irish for "dog". But Irish has no indefinite article, so "madra" can equally mean "dog" or "a dog". So now we know "an bhfuil madra agat?" is "is [a] dog ?".
Irish has prepositional pronouns; it combines prepositions (like "ag", which means "at") with pronouns (like "tú", which means "you"), to create one word (like "agat", which means "at you"). For some more examples: "agam" means "at me"; "aige" means "at him"; "romham" means "before me". So how does "is a dog at you?" mean "do you have a dog?"?
The language has no verb "to have" - it uses an "existential clause" to indicate possession instead. "I have a dog" is translated as "a dog is at me". Similarly, "you have a cat" would be "tá cat agat", and "Seán had bananas" would be "bhí bananaí ag Seán".

Maybe it makes some sense now that the phrase "do you have a dog?" is "an bhfuil madra agat?" (literally "is a dog at you?") in Irish. I suppose Irish is a weird language too.

Language quirks like these mentioned above further complicate translating a simple "yes" or "no" answer. We saw previously that you need to analyse the verb in the question before you give an answer, but now we see that the question verb might not be the same in the two languages; the main verb of "do you have a dog?" is "have" in English, but "is" in Irish. So even if all you want to do is translate the word "yes", you'll first have to translate whatever question you are trying to answer!

Sidenote: the copula

Irish has this thing called the copula, which is kind of like a verb but isn't actually a verb. Lots of yes/no questions asked are based on this, and are answered in a different way to verbal questions.

The copula is used to express identity or equivalence, but I'll leave you to read the wikipedia bit about it linked above for a better explanation. I'll just give a few examples of its usage:

English	is Máire a doctor?	yes	no
Irish	an dochtúir í Máire?	'sea	ní hea
Literal translation	is a doctor her Máire?	are they	aren't they

A slightly different question changes the question and answer quite a bit:

English	is Máire the doctor?	yes	no
Irish	an í Máire an dochtúir?	is í	ní hí
Literal translation	is her Máire the doctor?	is her	isn't her

And here's a more complicated usage:

English	would you like a drink?	yes	no
Irish	ar mhaith leat deoch?	ba mhaith	níor mhaith
Literal translation	would be good with-you a drink?	would be good	wouldn't be good

How can a computer do this?

Fluent Irish speakers can do all of the above without thinking about it. But it's difficult to get a computer to do it.

Guided by Dr. Teresa Lynn, I made progress on a tool to do this. Following is the steps taken.

First, I compiled a small corpus of yes/no questions and answers in English and Irish. I used this to help develop and to test the tool.
You give the tool a question in Irish that you would like to answer with a "yes" or "no", by eg. running the command echo "an bhfuil sí ceart go leor?" | python3 process.py
Dr. Elaine Uí Dhonnchadha wrote a part-of-speech tagger for Irish that is run by this tool.
The output of the POS-tagger is converted into CoNLL-U Format.
The CoNNL-U data is passed to Maltparser, trained on the Irish dependency treebank. This adds grammatical annotations to the question, showing the relations between words.
The tool can now analyse the grammar of the question, and decide if it is a verbal question, or is based on the copula^[1].
- If the question is verbal, the tool tries to gather information about the lemma and tense/aspect/mood of the verb.
- If the question is copular, the tool tries to find the copula form used^[2], and its lemma. It also tries to find the associated predicate, and its lemma.
It outputs a report based on this information, that can be used to generate a response to the question^[3].

Here's an example output of the tool:

{
    "type": "verb",
    "error": null,
    "verb": {
        "lemma": "buail",
        "surface": "mbuailfidh",
        "tams": [
            "FutInd"
        ]
    },
    "question": "Nach mbuailfidh Eilís an bithiúnach ?"
}

It tells you that the question, "Nach mbuailfidh Eilís an bithiúnach?" ("Won't Eilís hit the scoundrel?"), is verbal, based on the verb "buail" in the FutInd (future indicative) tense. Your answer should then also be the verb "buail" in the FutInd tense. The tool outputs all this in JSON, so it can be easily parsed by another tool to synthesise the verb response. I really want to put together this second tool soon, so that the full pipeline will be complete.

I learned so much in doing this project, and just as much again in doing the other projects across my internship. More importantly, I got to work with a language I love with some superb people, and I'm very thankful.

There's still no shortage of other work to do in Irish language tech technology (and no shortage of improvements to make to just this tool). Hopefully I'll get to do some more soon!

[1] Right now, the tool decides if the question is based on the copula by just looking at the root of the dependency tree, and seeing if the word there isn't a verb. Unfortunately, Maltparser sometimes incorrectly parses the question, and picks a verb in the question that shouldn't be the root to be the root (and sometimes giving a false-negative to the question being copula-based), or chooses a non-verb to be the root of the sentence where it should have chosen a verb (and giving a false-positive). Edit: I was suggested to change the word "top" here to "root", to maybe make this footnote more easily understood. I also need to note that the inaccuracy mentioned here is by no means the fault of Maltparser, but is due to the fact that the Irish treebank that Maltparser is trained on is very small, at just 1020 trees. Irish is a low-resource language! If/when there is more data, it will be more accurate.

[2] The tool tries to walk down the tree, finding the uppermost item tagged as being the copula. However, the part-of-speech tagger sometimes struggles with the copula ("an" is often marked as being a definite article instead of the copula, and "ar" is sometimes marked as a preposition instead of the copula, for example). To try to work around this, there's an explicit list of likely forms of the copula in a question that are also looked for, regardless of the part of speech reported for these words.

[3] This tool gives you the information that your answer should be, for example, the verb "ith" ("to eat") in the past tense. Actually synthesising this verb, ie. outputting "d'ith" ("ate") or "níor ith" ("didn't eat"), is a different problem. I've tried my hand at this problem, but it's a fair bit of work.

Kattis problem Pivot: Incrementally improving the performance of a python script, until nothing makes sense anymore

Noah Ó Donnaile — Wed, 11 Apr 2018 16:54:19 GMT

I have been doing a lot of competitive programming problems on Kattis lately. It is my favourite online judge, and I have been enjoying my climb up the Irish scoreboard in an attempt to regain my former glory of first place.

In particular, today I was tackling the problem Pivot. It is quite a small, simple, self-contained problem, but in trying to optimise my answer, I learned some interesting things.

Here is the gist of the problem statement:

An O(n) Partition algorithm partitions an array A around a pivot element (pivot is a member of A) into three parts: a left sub-array that contains elements that are ≤ pivot, the pivot itself, and a right sub-array that contains elements that are > pivot.

Now the actual job in this problem is this: Starting from an array A that has n distinct integers, we partition A using one of the member of A as pivot to produce a transformed array A’. Given this transformed array A’, your job is to count how many integers that could have been the chosen pivot.

Simply, we are given the output of a partition, and have to count the number of possible values in the list that could have been used as the pivot.

We are told that the number of elements, n, is in the range 3 ≤ n ≤ 100000. We also know that all input numbers are 32-bit signed integers (but this doesn't make that big of a difference when we're working in Python).

Now, let's go down the rabbit hole of solving this problem. And then let's go a little further.

Naïve Solution

I'm not saying it's your fault / Although you could have done more

When we break it down, we see that we are trying to find each number in the input list (which I will call D) such that D[i] is greater than or equal to all D[j] where j < i and less than all D[k] where k > i. This idea may be implemented like the following.

n = int(input())

d = list(map(int, input().split()))

ans = 0  
for i in range(n):  
    ok = True
    for j in range(i):
        if d[i] < d[j]:
            ok = False
            break
    for k in range(i+1, n):
        if d[i] > d[k]:
            ok = False
            break
    if ok:
        ans += 1
print(ans)

This solution has time complexity O(n^2), and it exceeds the 1 second time limit on the 4th test case on Kattis. It is simply not fast enough.

This is because, for every one of the (up to) one million input numbers, we are iterating over (in the worst case) every other one of the numbers. That's a lot of comparisons.

There are some minor optimisations you could do, like skipping looping over k if ok is already False, but I doubt these could squeeze it under the time limit. However, probably worth trying some optimisations if you have no other ideas, you're in the squeaky bum time of a programming competition, and you're desperate to get a few extra points.

O(n) fleek

I knew that O(n^2) wouldn't suffice when I read the question - 10^12 comparisons is too many to do in one second. However, I realised straight away that this problem could be solved in O(n) time.

You don't need to compare each number with every number before it to see if it is greater than or equal to each of them. You only need to compare the number to the maximum number before it. Likewise, you only need to compare the number to the minimum number after it.

You can build a list, which I call left below, of "the largest number to the left of this element", in O(n) time.

You can build an equivalent list right of "the smallest number to the right of this element" in similar fashion.

Reading the code is probably easier to understand than me rambling about it. Here is the first solution I submitted.

n = int(input())

d = list(map(int, input().split()))

left = [0 for _ in range(n)]  
left[0] = 0  
for i in range(1, n):  
    left[i] = max(left[i-1], d[i-1])

right = [2**32 - 1 for _ in range(n)]  
for i in range(n-2, -1, -1):  
    right[i] = min(right[i+1], d[i+1])

ans = 0  
for i in range(n):  
    if d[i] >= left[i] and d[i] < right[i]:
        ans += 1
print(ans)

Kattis told me this ran in 0.16 seconds, well below the time limit, and suitable to get full points for this problem. Job done!

Kattis has a nice feature that shows you a scoreboard of the ten fastest solutions to each problem, broken down by language. I often go and check out this table after I solve a problem, to see how my solution compares.

On peeking at the scoreboard for this problem, I was shooketh. Not only was the fastest Python 3 submission a mere 0.06 seconds, but it was submitted by my classmate/arch-nemesis/competitive programming teammate Cian Ruane. The horror! I couldn't live with his submission being a full tenth of a second faster; imagine how big his head would be if he saw this! I must do better.

Speedup #1

Jealousy's what the cheddar brings, for the cheddar it's anything goes

Quickly (I submitted the new solution just 1 minute 18 seconds later), I realised that I was doing twice the work I needed to. Instead of building the left list, I could just keep a running tally of the largest number seen so far while I'm iterating the list at the end to compute the answer. This will make the solution use around half as much space, will save an entire iteration of the array, and will surely improve my speed.

Maybe this is how Cian got a better result? Following is what I submitted.

n = int(input())

d = list(map(int, input().split()))

right = [2**32 - 1 for _ in range(n)]  
for i in range(n-2, -1, -1):  
    right[i] = min(right[i+1], d[i+1])

ans = 0  
maxn = 0  
for i in range(n):  
    if d[i] >= maxn and d[i] < right[i]:
        ans += 1
    maxn = max(maxn, d[i])
print(ans)

It ran in 0.14s. Not even fast enough to appear at 10th place on the scoreboard, never mind first.

From previous experience, I've seen that the most trivial Python problem runs in 0.02 seconds. I assume this is the cost of starting the Python interpreter, parsing the program, etc. Once a problem runs in < 0.05s or so, there is likely little space to speed it up. However, we're not at that limit yet. Besides, I can see someone else did it in 0.06s, and the rest of the scoreboard is at 0.07s - 0.09s, so it must be doable!

4 minutes older, 0.04s faster

2 minutes and 45 seconds later, I submitted the following:

n = int(input())

d = list(map(int, input().split()))

right = [2**32 - 1 for _ in range(n)]  
for i in range(n-2, -1, -1):  
    right[i] = right[i+1]
    if d[i+1] < right[i]:
        right[i] = d[i+1]

ans = 0  
maxn = 0  
for i in range(n):  
    if d[i] >= maxn and d[i] < right[i]:
        ans += 1
    if d[i] > maxn:
        maxn = d[i]
print(ans)

I have seen in several places people avoiding the builtin Python max and min, and looking at this problem, I don't see any other obvious potential speedups. It makes sense to me that these could be slower than simple if statements; not only is there the overhead of calling a function every time, but the functions also support getting the max or minimum values from an iterable, so must be some logic in there to figure out if that is necessary that we can avoid.

This solution ran in 0.12s.

Cian's solution still took half that amount of time. My curiosity was piqued; how did he do it? I decided to read his solution and find out - it could teach me a lot.

Peeking behind the curtain

I went to the github repo of his Kattis solutions and looked at his solution:

def main():  
    N = int(input())
    a = [int(x) for x in input().split()]
    mx = -1
    max_left = [None] * N
    for i in range(N):
        if a[i] > mx:
            mx = a[i]
        max_left[i] = mx
    mn = mx
    possible = 0
    for i in range(N - 1, -1, -1):
        if a[i] < mn:
            mn = a[i]
        if mn == max_left[i]:
            possible += 1
    print(possible)

if __name__ == '__main__':  
    main()

His solution is... more or less identical. He does his lists in the opposite order to me, but that shouldn't make a difference?

He has one fewer comparison in his final loop - maybe that's it? He builds his list of input values in a list comprehension instead of a map - that could be slightly more optimal? Maybe I'm missing something in his iteration orders, and there is some secret smart speedup going on?

I stole some of his ideas, and submitted the following.

n = int(input())

d = [int(x) for x in input().split()]

left = [None] * n  
maxn = 0  
for i in range(n):  
    if d[i] > maxn:
        maxn = d[i]
    left[i] = maxn

ans = 0  
minn = maxn  
for i in range(n-1, -1, -1):  
    if d[i] < minn:
        minn = d[i]
    if minn == left[i]:
        ans += 1
print(ans)

This ran in 0.09s.

WTF

Defeated, I fired him a message.

Noah: @Cian Ruane I have pretty much the exact same solution as you for problem pivot, but yours is 0.03s faster, wtf

Cian: That's incredible

...

Noah: do you get a magic speedup from the if __name__ == '__main__': line

Noah: did you get lucky with the server load or something

...

Cian: Try putting it into a function

Noah: surely that'd be slower

Cian: Maybe it's doing some sort of jit stuff or soemthing

Sure, why not try? I submitted the following.

def main():  
    n = int(input())

    d = [int(x) for x in input().split()]

    left = [None] * n
    maxn = 0
    for i in range(n):
        if d[i] > maxn:
            maxn = d[i]
        left[i] = maxn

    ans = 0
    minn = maxn
    for i in range(n-1, -1, -1):
        if d[i] < minn:
            minn = d[i]
        if minn == left[i]:
            ans += 1
    print(ans)

if __name__ == '__main__':  
    main()

This solution ran in 0.06s.

WTF 2: Electric Boogaloo

Why does putting the code in a function speed it up? Surely the overhead of jumping to a function should make the code slower, and not faster? Is Cian right, and that there's some JIT stuff going on?

No - but Stack Overflow has the answer:

Scharron: Just an intuition, not sure if it's true: I would guess it's because of scopes. In the function case, a new scope is created (i.e. kind of a hash with variable names bound to their value). Without a function, variables are in the global scope, when you can find lot of stuff, hence slowing down the loop

katrielalex: @Scharron you're half correct. It is about scopes, but the reason it's faster in locals is that local scopes are actually implemented as arrays instead of dictionaries (since their size is known at compile-time).

katriealex then goes into more interesting details about variable scopes in a full answer.

ecatmur also looks at the bytecode and shows how it comes out in the interpreter.

Super interesting.

Postmortem

Having seen all of this, what have I learned?

There is no big downside, but a potential upside, to putting my scripts inside a function, instead of in the global scope.
- It is faster, as I saw here. This speedup I guess becomes less obvious when there are fewer variable accesses in the script.
- Some of the answers here mention benefits of the code not being run on module import.
- It creates cleaner code (in my opinion). I was just too lazy to do it for throwaway competitive programming solutions before.
I should consider using an if instead of max and min if I am only comparing 2 values, and I am worried about performance.
- The code is not as nice, but it could save valuable time if it is run often.
I should learn off this page about python performance tips that katriealex linked. There are some great details about the innards of Python there.
I should keep on Cian's good side, so that he continues to be on my team and not on a competing team. That said, I have done better than him in some, previous, competitions, and I will never let him forget those.

My performance at the Student Battle Dev Season 11

Noah Ó Donnaile — Wed, 28 Mar 2018 16:14:54 GMT

I had a lot of fun yesterday evening competing in the 11th edition of the #battledev competition. I hadn't taken part before, but decided it might be worth checking out what competitions happen beyond Ireland's shores, and a French friend recommended this one.

The competition is French, but it allows anyone from an EEC country to compete. It lasts two hours, and consists of six questions. You must solve the first question before you can see the second question, and so on. The winner is the first person to solve the last question, then the second person to solve it, and so on for everyone who solved the last question, followed by the first person to solve the second last question, etc.

tá imní orm #battledev pic.twitter.com/O5Agy2eA8M
— noah ö donnaile (@iandioch) 27 Márta 2018

The first 3 questions were in miscellaneous topics, but were all reasonably easy.

Problem allyoucaneat required adding given %-discounts to orders of given sizes, and outputting the final sum of transactions.
tripadvisor involved finding the location from the input with the highest average rating, based on its rating in three subcategories, and outputting this average.
Then rocky13 asked you to find your k friends who's ratings for previous films most closely matched your own rating (by sorting by their minimum total difference in ratings), and output the average of their ratings for a new film.

I did quite well at these.

Happy with how I did anyway. Some screenshots I took to prove I'm not terrible: pic.twitter.com/2EkGqHfMn2
— noah ö donnaile (@iandioch) 27 Márta 2018

Then came pancakes, which was a Towers-of-Hanoi-like problem. The inputs were small enough to brute force the optimal solution with a tiny bit of tree-pruning.

Problem #5 was tramways12, and it was much more difficult. The input described a circle of vertices 0 to N-1, and a matrix of the number of people who wanted to get from each vertex to each other vertex. You were asked to output the maximal possible number of people you could take by building edges between nodes, with the following caveats:

Each vertex can only touch at most one edge.
No edge connecting vertices A and B can intersect the edge connecting vertices C, and D. The input essentially described a regular N-gon, and you could not draw lines between corners of the N-gon that would intersect.

The solution needed to be O(N^3).

I realised you could use dynamic programming. The optimal arrangement of edges within the bounds of vertices [A, B] was independent of the arrangements of edges at vertices outside of this range. However, I found that the implementation of this dynamic programming was easier said than done.

I tried many non-working solutions, that missed edge cases, or that accidentally didn't try some arrangement of vertices, but in the last minute of the competition, I uploaded my final solution. It evaluated... and evaluated...

Seeing as the submission system was flaky (and went offline a few times during the contest, under heavy load), I re-submitted.

Around the final second of the contest, it finished evaluating, and told me my solution was correct.

My #battledev solution to problem 5 finished executing in the last second of the contest, idk if my points will be counted. Suspenseful! pic.twitter.com/dg5dEil9Rq
— noah ö donnaile (@iandioch) 27 Márta 2018

Although I emailed the judges to enquire about this, in the end it seems they decided my solution didn't count. Oh well! It was an exciting end to the competition.

While I didn't get to see it during the competition, afterwards I read ducks7, the sixth and final problem, which appeared to be some form of duck-themed max-flow.

The following day (today), they released the results, in a blog post, and via email.

Bonjour,

Merci d'avoir participé à la Battle Dev.  
Les résultats définitifs de la saison 11 sont désormais connus.

Vous êtes 41eme sur 3322 au classement général.

There were 3322 competitors, of whom 2385 solved at least one problem. Out of that, I came 41st! I'm delighted. Had my final submission been accepted, I would have jumped up to 29th place, but as they say, c'est la vie.

However, it was pointed out in the blog post, that Dublin City University was ranked as the #2 university in the competition! I guess many of the people above me in the rankings were not representing a university.

hahahaha, unexpectedhttps://t.co/SKGkV2KfZZ @dcucomputing pic.twitter.com/k0m9FNwEXX
— noah ö donnaile (@iandioch) 28 Márta 2018

All in all, it was a fun competition, a little different to most I take part in. I was expecting to need to translate problems from French, but in the end they provided links to PDFs in English.

It seems bizarre to see DCU ranked highly in a French competition; I have saved the screenshot for future laughs. Let's see if we reach #1 next time!

In 2018, we finally won the IrlCPC

Noah Ó Donnaile — Sun, 11 Mar 2018 15:53:27 GMT

The Irish Collegiate Programming Contest (IrlCPC) is an annual programming competition hosted in University College Cork.

I have attended the IrlCPC before, finishing second in 2016 and third in 2017, delighted both times. This year, there were seven problems on the list, roughly arranged in order of difficulty; the first six were worth 10 points each, the final one worth 11. We found the fifth problem, which was based on a version of the Countdown numbers game, the most difficult.

I was on a team called -=[B]ichael [B] [B]iggins=- with my friends Cian Ruane and Ciara Godwin. We practised a fair amount over the past few months on Kattis, so we weren't altogether unprepared, although it did feel like that on the morning.

We achieved a perfect score of 71 points 63 minutes or so before the end of the contest, nabbing us first place in the university section.

Congrats to Ciara Godwin, Cian Ruane, & Noah Ó Donnaile of @DublinCityUni. Winners of the 2018 Irish Collegiate Programming Competition at @UCC from a field of 40 teams. DCU also has 2nd place! Event sponsored by @Google, @PoppuloTech and @insight_centre. #IrlCPC @muirtheimhne pic.twitter.com/6IJVnGG2sk
— Prof. Barry O'Sullivan, MRIA (@BarryOSullivan) 10 Márta 2018

We were emblazoned with Swiss flags because I had just arrived the night before from Zürich, where I'm interning at Google. This is the first time I've had a team uniform at a programming competition.

Congrats to Ciara, Cian & Noah on this great achievement!
Congrats also to our 2nd placed team.
Very good news for @dcucomputing! @insight_centre Thanks to @UCC for hosting. https://t.co/PKMoCLubL7
— Brian MacC (@muirtheimhne) 10 Márta 2018

It's worth pointing out that just like in 2016, a secondary school team (ie. in a different bracket) beat us to the mark with a perfect score after only one hour and 45 minutes: more than a full hour before we got there.

Our CPSSD comrades came second in the university division, also with a perfect score.

And the winners are...
#tech #ireland #cork #competition #IrlCPC pic.twitter.com/MssAU5Xkeu
— UCC ACM Student Chap (@uccacm) 10 Márta 2018

HackTrinity CTF Solutions

Noah Ó Donnaile — Sat, 03 Feb 2018 10:36:08 GMT

This week I took part in the HackTrinity CTF. It was my first CTF, but by the end of the week I managed to get all the flags. I think I may have been the second person to do it :)

Following is my solutions to each of the problems, in the order I solved them. The ones closer to the end may be more interesting...

Start Here

Welcome to HackTrinity '18.

If you've never played a Capture-The-Flag before, the objective is to obtain flags of the form HackTrinity{} by solving various challenges. That might involve exploiting a vulnerability in a website or finding it hidden in a file. Every time you submit a correct flag you get points. You can check your answer in the field below the challenge description.

Enter the flag HackTrinity{beginners_welcome} in the field below for 50 points.

Netopian

I vaguely knew in the back of my head that some Eircom router passwords could be cracked. After some Googling, I confirmed it.

Just input 1645 1200 into here, and get the flag C8CBE3D384E8F0F0124978FF00 back.

NotNotPetya

If you follow the link at the bottom to tinyurl.com/notnotpetya, it redirects to https://notnotpetyawebsitesupport4youtodayhere2serveyoudecryptedfiles.hacktrinity.me/.

Under the "Pay by Bitcoin" heading, there are the following instructions:

Step 1

Go to coinbase.com and sign up for an account

Step 2

Purchase $3000 BTC

Step 3

Send BTC to 1BYg2ZPPAk4SV251S1aAwq8oEAuVv1ZDUo

If you google 1BYg2ZPPAk4SV251S1aAwq8oEAuVv1ZDUo, it will surface this tweet.

The tweet's author is the flag.

Lenny

Problem statement:

You're a TA assigned with the task of correcting programming assignments. For this particular assignment the students were allowed to use any language of their choosing. Lenny has decided to use some obscure language and refuses to give details.

See if you can run it.

Attached is a file full of various lennys. I straightaway posited that this was a variation of brainfuck, but didn't know how to go about decoding it. Fortunately, if you just google "lenny esolang", this page is returned. It contains the following table to translate this back to a variant of brainfuck:

brainfuck	lennyfuck
+	( ͡° ͜ʖ ͡°)
-	(♥ ͜ʖ♥)
.	(> ͜ʖ<)< td="">
,	ᕙ( ͡° ͜ʖ ͡°)ᕗ
<	(∩ ͡° ͜ʖ ͡°)⊃━☆ﾟ.*
>	ᕦ( ͡°ヮ ͡°)ᕥ
^	ᕦ( ͡° ͜ʖ ͡°)ᕥ
v	( ͡°╭͜ʖ╮ ͡°)
x	ಠ_ಠ
[	( ͡°(
]	) ͡°)

After doing some search-and-replace in TextEdit, I was left with:

-[------->+<]>-.[--->++++<]>+.++.++++++++.>-[--->+<]>-.-[--->+<]>+.---------.+++++.-----.+++++++++++.+++++.++.--[->+++<]>-.+.------------.[--->+++++<]>.[----->+++<]>.+++++.+++++.-----.-----.+++++++++++++++.+.+++++.+[->+++<]>.+++++.[-->+<]>--.[->++<]>-.[--->+<]>-.------------.[-->+<]>-.---[->++<]>-.++.--[--->+<]>--..+++[++>---<]>.+[--->+<]>+.+++++++.-.--------.+++++++++.[-->+<]>.>--[-->+++<]>.

I ran this in this online brainfuck interpreter, and was returned the flag HackTrinity{jk_I_did_not_d0_th3_ass1gnmen7}.

SecureLocker

Problem statement:

Trinity have launched their new SecureLocker™ document storage facility which uses high-tech cryptography (AES-CBC + PBKDFv2) to securely store documents. Can you break into the vault and retrieve the flag?

https://locker.hacktrinity.me

If you open up the URL https://locker.hacktrinity.me, you'll see a field to enter a 4-digit PIN.

4 digits is very brute-forceable. Open the site and paste the following into the Javascript console:

for (var i = 0; i < 10000; i++) {  
  var s = ("0000" + i).slice(-4);
  var result = tryPinCode(s);
  console.log(s, result);
}

After a while you'll see 8759 HackTrinity{your_encryption_is_only_as_strong_as_your_key_space} scroll past.

You can also just wait for it to finish, and then search for HackTrinity.

Whiteboard

Problem statement:

Say goodbye to Blackboard.... and hello to Whiteboard, the new Learning Management System for Trinity! All your grades are now available to view in a beautiful responsive interface.

Unfortunately you failed Ethical Hacking, so you don't get a Flag! Although if you manage to hack the system, your professor might give you some bonus marks...

https://whiteboard.hacktrinity.me

https://whiteboard.hacktrinity.me is the login page to a college website, with the credentials username=student1 password=password1 listed. There's a "Staff Area" tab at the top of the page. After you login, if you try to navigate to this tab, you'll be told you can't access it because you're not staff. Meanwhile, on the "My Grades" page, you see the notice:

Sorry, you have not passed all modules! To earn the flag you must pass all modules (get over 40%!).

If you look at the network tab in Chrome, you'll see that the cookie auth=student1%3Apassword1%3Astaff%3Dfalse is set when you log in as student1.

If you run the command document.cookie="auth=student1%3Apassword1%3Astaff%3Dtrue" in the Javascript console after logging in, you can access the "Staff Area" page, change your grade in ethical hacking (I changed mine to 101%), and then when you go back to the grades page, you'll see the flag HackTrinity{ThisWontWorkOnBlackboardSoDontEvenThinkAboutIt}.

EyeTee

Problem statement:

Trinity EyeTee have launched a new knowledge-base website! But can you get access to the super-secret password?

https://eyeteehelpdesk.hacktrinity.me/

Enter the flag as HackTrinity{}

This website is nothing but a search box, and different searches surface documents that contain that queried substring. However, an empty search returns previews of all documents, including the following:

**Super-secret password document** *STAFF*

In accordance with our password policy, the super-secret password is lower-case alphanumeric and 9 characters long. Password: a9j[...]

We now know the format and first three characters of the answer, a9j. If search try every possible lowercase letter and number appended to the end of this, you'll find that the next character is b; the answer starts with a9jb. If you continue this process you'll find the full answer is a9jb7ib39.

I did this manually and it didn't take long, but you could easily script it by iterating over all possible next characters for your given answer so far, and continuing this process with whichever resulting page has a larger size in bytes.

Koinex

Problem statement:

Being a white-hat hacker, you're invited by Koinex a cypto-exchange company to test their security, who knows if you are successful you might get some of that sweet Ethereum that you've been looking for.

https://koinex.hacktrinity.me

https://koinex.hacktrinity.me is a login page, with javascript username and password checking. The script to do that is as follows.

var _0xad25=["\x23\x75\x73\x65\x72\x6E\x61\x6D\x65","\x71\x75\x65\x72\x79\x53\x65\x6C\x65\x63\x74\x6F\x72","\x23\x70\x61\x73\x73","\x72\x65\x73\x75\x6C\x74\x73","\x67\x65\x74\x45\x6C\x65\x6D\x65\x6E\x74\x42\x79\x49\x64","\x61\x75\x74\x68\x2D\x62\x75\x74\x74\x6F\x6E","\x63\x6C\x69\x63\x6B","\x70\x72\x65\x76\x65\x6E\x74\x44\x65\x66\x61\x75\x6C\x74","\x76\x61\x6C\x75\x65","\x61\x64\x64\x45\x76\x65\x6E\x74\x4C\x69\x73\x74\x65\x6E\x65\x72","\x55\x73\x65\x72\x6E\x61\x6D\x65\x20\x6F\x72\x20\x50\x61\x73\x73\x77\x6F\x72\x64\x20\x63\x61\x6E\x6E\x6F\x74\x20\x62\x65\x20\x65\x6D\x70\x74\x79","\x70\x69\x72\x61\x74\x65\x73","\x59\x6F\x75\x27\x72\x65\x20\x6C\x6F\x67\x67\x65\x64\x20\x69\x6E\x21\x0A\x46\x6C\x61\x67\x20\x69\x73\x20\x48\x61\x63\x6B\x54\x72\x69\x6E\x69\x74\x79\x7B\x26\x6C\x74\x3B\x70\x61\x73\x73\x77\x6F\x72\x64\x26\x67\x74\x3B\x7D","\x57\x72\x6F\x6E\x67\x20\x43\x72\x65\x64\x65\x6E\x74\x69\x61\x6C\x73","\x64\x38\x61\x39\x37\x39\x34\x65\x36\x35\x38\x62\x38\x32\x64\x38\x30\x35\x62\x38\x37\x31\x39\x61\x62\x64\x34\x32\x64\x32\x63\x32\x63\x65\x39\x35\x36\x64\x31\x64","\x69\x6E\x6E\x65\x72\x48\x54\x4D\x4C","","\x73\x65\x74\x54\x69\x6D\x65\x6F\x75\x74"];var userField=document[_0xad25[1]](_0xad25[0]);var passField=document[_0xad25[1]](_0xad25[2]);var resultsDiv=document[_0xad25[4]](_0xad25[3]);var submitButton=document[_0xad25[4]](_0xad25[5]);submitButton[_0xad25[9]](_0xad25[6],function(_0xe58cx5){_0xe58cx5[_0xad25[7]]();tryAuth(userField[_0xad25[8]],passField[_0xad25[8]])});function tryAuth(_0xe58cx7,_0xe58cx8){if(!_0xe58cx7&&  !_0xe58cx8){invalidCredentials(_0xad25[10]);return};if(_0xe58cx7=== _0xad25[11]&& isValid(_0xe58cx8)){showSuccess(_0xad25[12])}else {invalidCredentials(_0xad25[13])}}function isValid(_0xe58cx8){return sha1(_0xe58cx8)=== _0xad25[14]}function invalidCredentials(_0xe58cxb){resultsDiv[_0xad25[15]]= _0xad25[16];window[_0xad25[17]](function(){resultsDiv[_0xad25[15]]= ``+ _0xe58cxb+ `
`},100)}function showSuccess(_0xe58cxd){resultsDiv[_0xad25[15]]= ``+ _0xe58cxd+ ``}

If you put that into jsbeautifier, it will show you that the script looks like this:

var userField = document['querySelector']('#username');  
var passField = document['querySelector']('#pass');  
var resultsDiv = document['getElementById']('results');  
var submitButton = document['getElementById']('auth-button');  
submitButton['addEventListener']('click', function(_0xe58cx5) {  
    _0xe58cx5['preventDefault']();
    tryAuth(userField['value'], passField['value'])
});

function tryAuth(_0xe58cx7, _0xe58cx8) {  
    if (!_0xe58cx7 && !_0xe58cx8) {
        invalidCredentials('Username or Password cannot be empty');
        return
    };
    if (_0xe58cx7 === 'pirates' && isValid(_0xe58cx8)) {
        showSuccess('You\'re logged in!\x0AFlag is HackTrinity{<password>}')
    } else {
        invalidCredentials('Wrong Credentials')
    }
}

function isValid(_0xe58cx8) {  
    return sha1(_0xe58cx8) === 'd8a9794e658b82d805b8719abd42d2c2ce956d1d'
}

function invalidCredentials(_0xe58cxb) {  
    resultsDiv['innerHTML'] = '';
    window['setTimeout'](function() {
        resultsDiv['innerHTML'] = `` + _0xe58cxb + ``
    }, 100)
}

function showSuccess(_0xe58cxd) {  
    resultsDiv['innerHTML'] = `` + _0xe58cxd + ``
}

This statement tells us the flag is the password:

if (_0xe58cx7 === 'pirates' && isValid(_0xe58cx8)) {  
    showSuccess('You\'re logged in!\x0AFlag is HackTrinity{<password>}')
}

We know from below that the password is some string which has the SHA-1 hash of d8a9794e658b82d805b8719abd42d2c2ce956d1d.

We can simply look this up in a rainbow table. I used this website, as it was the first result in Google. In one fifth of a second, it told me that the string banter fit this critera.

The flag is therefore HackTrinity{banter}.

Jar of Java

Problem statement:

Hey, check out this password checker that I wrote for my programming class! I think it's very secure but I'm not sure. Can you reverse-engineer it to make sure it can't be broken?

You can run the program using the command java -jar password.jar

You are given a file password.jar. Uploading it to an online decompiler returns the file PasswordCheck.java, which looks like this:

import java.util.Scanner;

public class PasswordCheck { public PasswordCheck() {}  
  public static String password = "HackTrinity{When_You_Lose_The_Code_You_Lose_Your_Secrets}";

  public static void main(String[] paramArrayOfString) {
    Scanner localScanner = new Scanner(System.in);
    System.out.printf("Enter password: ", new Object[0]);
    String str = localScanner.nextLine();
    if (str.equals(password)) {
      System.out.println("Congrats, that was the flag!");
    } else {
      System.out.println("Oops, that's not the flag!");
    }
  }
}

The flag is HackTrinity{When_You_Lose_The_Code_You_Lose_Your_Secrets}.

Lotto

Problem statement:

In an effort to raise funds, Trinity have launched a lottery!

See if you can predict the next lotto numbers and win the jackpot.

https://lotto.hacktrinity.me

lotto.hacktrinity.me is a site that gives you the previous lotto number, and gives you an input box to guess the next number. If you get it correct, you will be shown the flag. If you get it wrong, it will show you what the next number was. After 3 numbers, it tells you the game is over, and to try again (after a CAPTCHA).

However, it also gives you the implementation of the lotto number generator:

import java.util.Random;

public class GenerateLottoNumbers {  
  /* Output 1st, 2nd and 3rd Lotto numbers to stdout.*/
  public static void main(String[] args) {
    Random r = new Random();
    System.out.println(r.nextInt());
    System.out.println(r.nextInt());
    System.out.println(r.nextInt());
  }
}

java.util.Random is predictable though! After 2 outputs from r.nextInt(), you can guess the seed being used, and from there calculate the next output. Lery on StackOverflow wrote a script to do this. I just copied this script into ideone.com, put in the first 2 numbers from the lotto site, and ran it.

It outputted the third number, which when put into the lotto site returned the flag HackTrinity{they_call_it_pseudo_random_for_a_reason}.

Capture the Packet

Problem statement:

We (an unnamed three-letter agency) have managed to install a wire tap on a target's internet connection and have created the attached Packet Capture (PCAP) file.

Can you retrieve any useful information? It's probably all encrypted nowadays anyways thanks to those dratted tech companies.

Attached is a PCAP file. I uploaded this to packettotal.com to look at it. I looked at the transferred files, and opened the one in format text/plain. It contained the following string:

username=l33th4x0r&password=HackTrinity{this_is_what_the_NSA_sees_when_you_dont_encrypt}

The flag is HackTrinity{this_is_what_the_NSA_sees_when_you_dont_encrypt}.

Biccies

You are an investigative reporter with the Student Tribune and have filed a Freedom of Information Request for details of government spending on biscuits. However, the authorities have redacted the documents, citing National Security concerns. Can you find out how much they spent on Rich Tea?

Enter the flag as  
HackTrinity{} , including currency and commas where appropriate

Attached is a PDF with a table of biscuit types to prices. However, there is a black box over the prices.

There is an easier way, but I was limited to a corp computer, so couldn't install any software. I didn't have any luck with online PDF readers or editors- none were buggy in the way that I hoped, by not showing the black boxes. All either crashed, or correctly displayed/converted the document.

I read some of the PDF spec, figured out the formatting of a PDF document.

This post gives a very quick format of a PDF file.
This post actually goes line-by-line through an example PDF, explaining everything that is going on.
This post helped when I assumed a MediaBox was a big black redaction box.

The PDF was a mess, but from StackOverflow I learned about qpdf. sudo apt-get install qpdf on my VPS nabbed it, and then qpdf --qdf --object-streams=disable biscuitdoc.pdf out.pdf. I downloaded out.pdf back to my laptop, and opened it in TextEdit.

I took a look at it, tried to find where the text would be. There was still some weird stuff going on, but by far the longest object in the PDF started like this:

9 0 obj  
<<  
  /BitsPerComponent 8
  /ColorSpace /DeviceGray
  /Height 3488
  /Subtype /Image
  /Type /XObject
  /Width 2464
  /Length 10 0 R
>>
stream

This was followed by a lot of mostly-random characters, most of them being ^. I assumed this was the text in the document, and that the ^s were empty space or similar. Just to see what would happen, I deleted a few lines of them, saved the document, and opened it in Chrome. It looked like the following:

The text moved, but the black boxes didn't. Still, the values are visible. The flag is HackTrinity{€17,278,289}.

I still amn't totally sure how the whole PDF format works, but I know a lot more than I did before.

Tayto Smugglers

In the post-Brexit economic apocalypse of the distant future (2019), Tayto crisps are a highly sought after luxury. They are now subject to high customs and charges when they are exported to the UK. An illegal Tayto smuggling ring has cropped up, and the EU would like to crack down on it.

You've been tasked with searching all digital files at the (hard) border and have come across this innocuous-looking picture of the Campanile. Can you ensure it contains no hidden bag of Tayto's crisps?

Attached is a huge (1.2MB!) PNG of with a picture of TCD. This PNG does not load in some programs on my Mac laptop, but file tells me this:

campanile.png: PNG image data, -2555936 x 1067590, 73-bit interlaced

I expected there was another file type buried in there, because I knew it was very feasible to hide things in PNGs, and this PNG was such a large file for such a normal picture. I tried unzip-ing it, but to no avail. I opened up the file in an online hex editor. The words "Created with GIMP" were near the top of the file, quite clearly :)

After reading the hex and googling, I figured out that the file had both a PNG header (with the PNG magic number at the beginning, the PNG header in the middle, and the PNG footer at the end), and a JFIF/JPEG image (with the JFIF/JPEG magic number following the PNG magic number, and the JPEG footer just above the PNG header).

I tried renaming the file as a .jpg and opening it, but it still looked the same. I sat stumped for a long time. file still only recognised the file as a PNG, so I tried deleting the PNG magic bits from the beginning. file now said this:

campanile_clean.jpg: JPEG image data, JFIF standard 1.01, resolution (DPI), density 72x72, segment length 16, comment: "Created with GIMP", progressive, precision 8, 1000x1000, frames 3

I opened the image again, and there was a packet of Taytos looking at me, with the flag HackTrinity{you_can_pry_my_taytos_from_my_cold_dead_hands} underneath in the image.

I guess my image viewer was originally ignoring the JPG suffix and reading the file as a PNG anyway, and that's why it looked the same. I won't make the same mistake again.

To Catch a Cheater

College has been made aware of an exam paper leak. A copy of the printed paper was obtained and IT Services has been tasked with tracking the cheater. They believe it was printed on college printers. Can you help give the culprit a dose of r e a l i t y ?

Attached was a zip with 3 directories: Printer_1_228406, Printer_2_318008, and Printer_3_615028. Within each were 3 files, all of the same size, with a list of timestamps and usernames. These were the printer logs for the dates 31st March 2017, 1st April 2017, and 2nd April 2017.

Attached also was a PDF of the exam paper that was found.

After floundering, trying to look up details of the paper to see if someone was mentioning it online, I eventually realised that the PDF attached was not the exam paper, but a scan of the print-out of the exam paper. I looked at it more closely, and noticed little yellow dots. Ah!

Instructables had an article on how to make the dots more visible. After converting the PDF to a TIFF online, I downloaded imagemagick and ran convert -channel RG -fx 0 ExamPaper.tiff blue.png on my VPS, then downloaded blue.png to look at on my laptop. The dots were a little clearer.

The EFF has a fantastic online decoder for this pattern, including some explanation of the format. I entered the pattern I saw there (wrongly it turns out, but the tool could use the parity bits to fix it):

Parity mismatch for row 2.  
Parity mismatch for column 7.  
Correctable error at row 2 and col 7  
Making correction and processing corrected matrix:

           111111
  123456789012345
7   oo   oo  o  o  
6 o        o o  
5  o       o   o  
4        o o ooo  
3     o    oo o o  
2 oo  o o  o  oo  
1          o  oo  
0 oo  oo o o  ooo  
Printer serial number: 318008 [or 55318008]

Date: April 1, 2017

Time: 13:37

So we know that a printer with code 318008 or 55318008 (ie. Printer 2) printed this document on the 1st April 2017 (which we have the log for) at 13:37 (of course). The section of this log looks like this:

[01/Apr/2017:13:35:13] Alloway
[01/Apr/2017:13:35:49] DiFrancesco
[01/Apr/2017:13:37:03] wen-huat
[01/Apr/2017:13:37:40] FerrisBueller
[01/Apr/2017:13:38:12] Baert
[01/Apr/2017:13:38:17] Bdale
[01/Apr/2017:13:38:48] Anamaria

Only two people printed at this time. I tried wen-huat, which was wrong, but the flag did indeed turn out to be HackTrinity{FerrisBueller}. Gratifying!

Hugs and Kisses

Problem statement:

Get some hugs and kisses from this binary file.

Attached is a file called haxor. Opening it up in vim, I saw the ELF magic number straight away at the start, and some vaguely human-readable code near the end of the file. The program doesn't run on my macbook, but it runs fine on my Ubuntu VPS. I tried it:

me@my_vps ~ % ./haxor  
Usage ./haxor

Running it with any arg I tried tells me what's up:

me@my_vps ~ % ./haxor testflag  
This is not the flag :)

To a disassembler we go, to try to figure out what is the flag, or how does it check if what we entered is correct.

Playing with an online disassembler for a long time, I started to see the layout of the program. The graph view of the main function in that online disassembler really helped figure out the control flow.

This is what main looks like.

0x4006bd                  push   %rbp  
0x4006be                mov    %rsp,%rbp  
0x4006c1                push   %rbx  
0x4006c2                sub    $0x18,%rsp  
0x4006c6                mov    %edi,-0x14(%rbp)  
0x4006c9               mov    %rsi,-0x20(%rbp)  
0x4006cd               cmpl   $0x2,-0x14(%rbp)  
0x4006d1               je     0x4006f6   
0x4006d3               mov    -0x20(%rbp),%rax  
0x4006d7               mov    (%rax),%rax  
0x4006da               mov    %rax,%rsi  
0x4006dd               mov    $0x4007d4,%edi  
0x4006e2               mov    $0x0,%eax  
0x4006e7               callq  0x4004f0   
0x4006ec               mov    $0x1,%edi  
0x4006f1               callq  0x400530   
0x4006f6               mov    -0x20(%rbp),%rax  
0x4006fa               add    $0x8,%rax  
0x4006fe               mov    (%rax),%rbx  
0x400701               mov    $0x0,%eax  
0x400706               callq  0x400646   
0x40070b               mov    %rbx,%rsi  
0x40070e               mov    %rax,%rdi  
0x400711               callq  0x400510   
0x400716               test   %eax,%eax  
0x400718               jne    0x400726   
0x40071a               mov    $0x4007e5,%edi  
0x40071f               callq  0x4004e0   
0x400724              jmp    0x40073a   
0x400726              mov    $0x4007f4,%edi  
0x40072b              callq  0x4004e0   
0x400730              mov    $0x1,%edi  
0x400735              callq  0x400530   
0x40073a              mov    $0x0,%eax  
0x40073f              add    $0x18,%rsp  
0x400743              pop    %rbx  
0x400744              pop    %rbp  
0x400745              retq

After much study, it became clear that computeFlag (called at 0x400706 above) creates the flag, and then the program jumps (from 0x400711 above) to 0x400510 (ie. strcmp) where this flag is compared with the inputted flag. From there, it jumps around and prints different strings depending on whether the input flag was correct or not.

I opened gdb and set a breakpoint at 0x400510. I ran the program with the argument testflag, until it hit that breakpoint.

I could see that the arguments were at the locations of registers rsi and rdi (or more correctly rbx and rax), as these are set at 0x40070b and 0x40070e above, before strcmp is called at 0x400711. I printed the strings at these locations in gdb:

(gdb) x/s $rsi
0x7fffffffe782: "testflag"  
(gdb) x/s $rdi
0x602010:       "OTYDIHMRWBAFKPUZYDINSRWBGLKPUZ"

OTYDIHMRWBAFKPUZYDINSRWBGLKPUZ looks like our flag. And sure enough:

me@my_vps ~ % ./haxor OTYDIHMRWBAFKPUZYDINSRWBGLKPUZ  
Nice job. XOXO

HackTrinity{OTYDIHMRWBAFKPUZYDINSRWBGLKPUZ} gets me the points.

This all took much longer than it seems here, including re-learning lots of assembly, translating between AT&T and Intel assembly syntax, and looking at random code locations in both my online disassembler and gdb :)

Unhackable

Problem statement:

According to the owner, this site is literally unhackable.

https://unhackable.hacktrinity.me/

This web address just returns a page with the following HTML:

  
  
  
This site is literally unhackable. It's deployed as a static site. Try your best punks, but there's no way you're getting my flags.txt.

I spent so long on this! I did subdomain bruteforce searches, portscanned it, URL fuzzed, analysed the header files, looked at my cookies, everything I could think of. After hours, an idea popped into my head. The solution? https://unhackable.hacktrinity.me/.git/ is accessible.

For some reason it always redirected me to 127.0.0.1/.git/* after downloading whatever was at https://unhackable.hacktrinity.me/.git/*, but that's ok - we have all we need.

This excellent blogpost walked me through the steps of nabbing flags.txt from this .git. If we download the refs/heads/master file, we can see the hash of the latest commit (4ae239c8f94d1bee157117a2c74720f51d16a1bf). We can create an empty git repo locally, download this file and put it where it should be (.git/objects/4a/e239c8f94d1bee157117a2c74720f51d16a1bf), then read it.

$ git cat-file -p 4ae239c8f94d1bee157117a2c74720f51d16a1bf
tree 3c8848f3d42b50fd87bdf0045b07c73e9a2cbf69  
parent aebaab81b38bdb74cdc2e111118621d2f7dd001a  
author Rory Flynn  1517251354 +0000  
committer Rory Flynn  1517251354 +0000

Oops, remove those flags.txt. That was a close one

Presumably, this commit deleted flags.txt. We can see the parent's commit hash is aebaab81b38bdb74cdc2e111118621d2f7dd001a. We can download that commit (unhackable.hacktrinity.me/.git/objects/ae/baab81b38bdb74cdc2e111118621d2f7dd001a) and do the same as before.

$ git cat-file -p aebaab81b38bdb74cdc2e111118621d2f7dd001a
tree 0a3725964eaad4d19e4b422a8bbe1cf40e5a451e  
author Rory Flynn  1517251255 +0000  
committer Rory Flynn  1517251255 +0000

initial commit of flags and index.html

Cool! We can download the tree and read it:

git cat-file -p 0a3725964eaad4d19e4b422a8bbe1cf40e5a451e  
100644 blob c2440c908a67b4266b6be5690fab86ee25977d80    flags.txt  
100644 blob be6392988371f35dcc5dd6603e567e8153a70920    index.html

And now we can download the flags.txt blob and read it:

$ git cat-file -p c2440c908a67b4266b6be5690fab86ee25977d80
HackTrinity{hidden_git_directorys_gonna_git_ya}

Heyo.

Unfurlme

Problem:

Can you reverse engineer this obfuscated PHP source code?

https://unfurlme.hacktrinity.me/

This page has the HTML source:

UnfurlMe.phpEnter the password:

View Source

This isn't that interesting. Just a password input box, and a View Source hyperlink. But if you click the View Source link on the page, it appends ?src to the URL, and shows you the PHP used to generate the page:



You can see here that the first line of this program just shows the PHP source code when ?src is present, and then halts the program. we can ignore this line.

I first put this script through an online PHP deobfuscator. It returned the following:

$wowzers="base64_decode";
$pandas="convert_uudecode";
$jenga="gzuncompress";
$a=$pandas(strrev($wowzers('CmAKSURCKCg5JzpIXUQzV11SNVMxNzkvClpZRDZBISc5Iik3O0otNTNQTVYzTClTOU9AJCxXODc2VCwmNFNAVzRWWCQyUS02NkElRDVJPVcrVzE1OU0KWl00MDRJRjxGOTQxVTFFOjdRRDAtPSM1QU1CLDoxRjMuTSQ1VTkjNjgxVTglQUUxKylXNUswNS5EUTQ2TQpSRCYtKFlGMjpRVjQsRTU1JyFVLUExIy5SSUY7NyVTOFRBJC4vXSYtTC1DNUQ9RDMwSUcxI0ElM0s0IztNCkYpRS0oMUY8WUBFODNdJjNSVFY6VigzNk8xRzwmNVQrOVE2PEtURjtQMUc9OEVXMUdNQjVYPVQ1UTkzPk0KNEk2PjNBJzlMLSM2Uy03PTY1RTNPSDc4NiU2NTZBRDpPNEc2UEEzPjIxRTBPKVMsUyw0NE0xVzlDKVY7TQooMTcuLkFFMS9NIjRWTFYxU0gkLCZBUyxYNVQ5WDQnM1lERDJUWTQsS0E3Lio9Rjk5PVY9WTlDPkUhJS1NCjJdIjQ5OUQ0Rj1XOSg5My44STYzUik2MFUtVz1LNFM1WClDMEFBVzNWMVYwRSknPFZIND5QTEQtKT0jLU0KTFVWLTUxNDhTLEcxOTVENktBVztXOTU5LD1GMlhANTQ2KUMyOkFGMzIxNjZYSFQ9SzFFPFIwRjxQLUc4TQpZTDQ5USglNE8xRzNLSEc8L0klPlYpNTwnWVQ7TDFUM1BgIzRWTUQ7IVFEPFlJJi5UTUYxMyUzPVEpJjJNCjktNz1TXDQ9R11GMVMsRjFGISU5JklUMFEoVzVCVVYtIU0kNFMsMzhUWEQ9UlRWPU4tRjpIUVQ4UDFEO00KV0FEODBFUzEuJTUzJkFVM0g1Nyw1MUc0RlFEOjpBUzsyLUMsRFk0PkotRTEhRVc6UC1TLC05UypEJUM9TQpSNVU7LEEzO0Q1RDtELTY9VzxTPDpBNjtFJSU9Vyk0OFoxRThCLTc0U0xSNCxNRD5aLTQ+Vzk0OSJdMjZNClM9JDRRRDYxNS0jPFJZRjolQUM8TlE2OFRVVjtUREY5WElGMUU1JS1KVSQ8SF1CNSxdJjQmQVYxJUk1MU0KT0RXLVQ0NT4tNUYxMVk0MVUlJjxYPVY4SDFHOi5NND0kMVMsU0VVOTFZJCxUWTQzWDFVMktMMjpSRFctTQooNTQsOCUjPFM5VSo1TUI7Mz1FOlYsNTU3NSY0MzFWKystJj1ZNFQ8JikzLFAkRDApISMtNz01OyspVSxNCiFNQjw4IUcsT0BUOE8oRzRKOUcsMlUmNEQ1VypSMSYzMylGNiNJJzxLQDMwJ0EjLCtdIi5QQVc9Ii1ENU0KOClHLVUhRTxLSSQtOTFUMkEpNiwyQSMsNi0lLDdFJTkjOVYzVyEzM0ItVy0lNVY6QjFWMFhdVCopJTYxTQpQISUxTDFVMyJBVTQwWTY7TEUmMiElJjpULTUuVjQzPDdBJT06QUMzLEk3PSRBRDpHIVU4Uy1XNTEtVjtNCkk1NTkwWUY+WkE2OFRBRTU3XVIqUD1ELFRVNDRJJVc6UiVTMyQtJjooUTYzJUlUPSlFVDQiVUQtVFw2LU0KLCU1LjhBNjRPSVY7QilTOktZVD0qNUYoSDQmOU8tNjlEXSUtVjRWPEEpJipTLTc5UiE3O08tRjtVSVc5TQ==')));
eval("eval($wowzers($jenga($pandas($wowzers($a)))));");  


I removed the function variables to see things more clearly:

$a=convert_uudecode(strrev(base64_decode('CmAKSURCKCg5JzpIXUQzV11SNVMxNzkvClpZRDZBISc5Iik3O0otNTNQTVYzTClTOU9AJCxXODc2VCwmNFNAVzRWWCQyUS02NkElRDVJPVcrVzE1OU0KWl00MDRJRjxGOTQxVTFFOjdRRDAtPSM1QU1CLDoxRjMuTSQ1VTkjNjgxVTglQUUxKylXNUswNS5EUTQ2TQpSRCYtKFlGMjpRVjQsRTU1JyFVLUExIy5SSUY7NyVTOFRBJC4vXSYtTC1DNUQ9RDMwSUcxI0ElM0s0IztNCkYpRS0oMUY8WUBFODNdJjNSVFY6VigzNk8xRzwmNVQrOVE2PEtURjtQMUc9OEVXMUdNQjVYPVQ1UTkzPk0KNEk2PjNBJzlMLSM2Uy03PTY1RTNPSDc4NiU2NTZBRDpPNEc2UEEzPjIxRTBPKVMsUyw0NE0xVzlDKVY7TQooMTcuLkFFMS9NIjRWTFYxU0gkLCZBUyxYNVQ5WDQnM1lERDJUWTQsS0E3Lio9Rjk5PVY9WTlDPkUhJS1NCjJdIjQ5OUQ0Rj1XOSg5My44STYzUik2MFUtVz1LNFM1WClDMEFBVzNWMVYwRSknPFZIND5QTEQtKT0jLU0KTFVWLTUxNDhTLEcxOTVENktBVztXOTU5LD1GMlhANTQ2KUMyOkFGMzIxNjZYSFQ9SzFFPFIwRjxQLUc4TQpZTDQ5USglNE8xRzNLSEc8L0klPlYpNTwnWVQ7TDFUM1BgIzRWTUQ7IVFEPFlJJi5UTUYxMyUzPVEpJjJNCjktNz1TXDQ9R11GMVMsRjFGISU5JklUMFEoVzVCVVYtIU0kNFMsMzhUWEQ9UlRWPU4tRjpIUVQ4UDFEO00KV0FEODBFUzEuJTUzJkFVM0g1Nyw1MUc0RlFEOjpBUzsyLUMsRFk0PkotRTEhRVc6UC1TLC05UypEJUM9TQpSNVU7LEEzO0Q1RDtELTY9VzxTPDpBNjtFJSU9Vyk0OFoxRThCLTc0U0xSNCxNRD5aLTQ+Vzk0OSJdMjZNClM9JDRRRDYxNS0jPFJZRjolQUM8TlE2OFRVVjtUREY5WElGMUU1JS1KVSQ8SF1CNSxdJjQmQVYxJUk1MU0KT0RXLVQ0NT4tNUYxMVk0MVUlJjxYPVY4SDFHOi5NND0kMVMsU0VVOTFZJCxUWTQzWDFVMktMMjpSRFctTQooNTQsOCUjPFM5VSo1TUI7Mz1FOlYsNTU3NSY0MzFWKystJj1ZNFQ8JikzLFAkRDApISMtNz01OyspVSxNCiFNQjw4IUcsT0BUOE8oRzRKOUcsMlUmNEQ1VypSMSYzMylGNiNJJzxLQDMwJ0EjLCtdIi5QQVc9Ii1ENU0KOClHLVUhRTxLSSQtOTFUMkEpNiwyQSMsNi0lLDdFJTkjOVYzVyEzM0ItVy0lNVY6QjFWMFhdVCopJTYxTQpQISUxTDFVMyJBVTQwWTY7TEUmMiElJjpULTUuVjQzPDdBJT06QUMzLEk3PSRBRDpHIVU4Uy1XNTEtVjtNCkk1NTkwWUY+WkE2OFRBRTU3XVIqUD1ELFRVNDRJJVc6UiVTMyQtJjooUTYzJUlUPSlFVDQiVUQtVFw2LU0KLCU1LjhBNjRPSVY7QilTOktZVD0qNUYoSDQmOU8tNjlEXSUtVjRWPEEpJipTLTc5UiE3O08tRjtVSVc5TQ==')));
eval("eval(base64_decode(gzuncompress(convert_uudecode(base64_decode($a)))));");  


I tried echo $a, to see what $a was before it was taken apart and evaluated, and saw that it contained the following:

gzuncompress(base64_decode("eJwNkk2bojoQhX9QL5o46MBSIIwJEMlHhcDO1rkqiQMt2Gp+/WVXtahzznPeUiocQWsscPgjHDuzLN8ZtXWq569SthaAHilmnPSXBOTlDPpEaI+OxCdbkeE7sbM0wOfCdYW0SV08R1baKDY4JkrPu6rXVCBwxp8/K08GA8+pzCZbSLdr+udPmR2vjRr/cH/2pXr+A3RKmWW40IBA012FsE9tcK/dSPeWUS6jWSn+U+Vsp1X1EH7y2i++KTxMNt0NQgYs34DuKNjthcgxpauENQFeMyU47y/EZEGhFPoLV/hpMj4UeFjxfi4omtalnr8Ejnrp3UEi1PGsY/BeFwyCzzKLS+3QsbbTzaBwtQemhZs77ucdnEdm8LoUrv1d+6M33pkyAFSjyNd23Ro8ZjLfRtU1uhOXFMQNG9PbHwnDpcLhjcnwm2vN4a33PKA7mbWr1CJFdPfFc3FoguO3usYHbqu1SFkt8jyrLAnKvP00ODloNGqRvxZOrz+NtoPR1eK9bsprd2rTkwJ8YdRNhZJ2VQX8JgLeVwoxkZEYFs3aDU7ml47I6K0yJ6preCdvOxaB2xW5+wsuAbrMjX96HgwfRFYP/R4Pez6ywgYfgJ9xk1NtJI9Lu8gEx38F0J3Gk6P+OFXN9tHobcgtmQC332oBTRy8pZu/jHVUaVaz/NUVussX3ldxSyjTy6qWGxV+gGyXvtpnm+qlY/EFrtoY26km2LoSbX9rdH6Rfl5+LXCFzPNGdV3l4oO8Htc1Wnjr84a7PGUYLSlZJnH4i2YLd9T+WrKFXEcTXX6uTKNNdZ2+aT7MBLWjTuEFfrjTAOzeTw/wiVAaYcqHN6Sx3Pc4Yv70H/g2lOkpMSjmrBdpaZNzetsW/wNOhhvH"))  


So let's unravel that by running:

base64_decode(gzuncompress(convert_uudecode(base64_decode(gzuncompress(base64_decode("eJwNkk2bojoQhX9QL5o46MBSIIwJEMlHhcDO1rkqiQMt2Gp+/WVXtahzznPeUiocQWsscPgjHDuzLN8ZtXWq569SthaAHilmnPSXBOTlDPpEaI+OxCdbkeE7sbM0wOfCdYW0SV08R1baKDY4JkrPu6rXVCBwxp8/K08GA8+pzCZbSLdr+udPmR2vjRr/cH/2pXr+A3RKmWW40IBA012FsE9tcK/dSPeWUS6jWSn+U+Vsp1X1EH7y2i++KTxMNt0NQgYs34DuKNjthcgxpauENQFeMyU47y/EZEGhFPoLV/hpMj4UeFjxfi4omtalnr8Ejnrp3UEi1PGsY/BeFwyCzzKLS+3QsbbTzaBwtQemhZs77ucdnEdm8LoUrv1d+6M33pkyAFSjyNd23Ro8ZjLfRtU1uhOXFMQNG9PbHwnDpcLhjcnwm2vN4a33PKA7mbWr1CJFdPfFc3FoguO3usYHbqu1SFkt8jyrLAnKvP00ODloNGqRvxZOrz+NtoPR1eK9bsprd2rTkwJ8YdRNhZJ2VQX8JgLeVwoxkZEYFs3aDU7ml47I6K0yJ6preCdvOxaB2xW5+wsuAbrMjX96HgwfRFYP/R4Pez6ywgYfgJ9xk1NtJI9Lu8gEx38F0J3Gk6P+OFXN9tHobcgtmQC332oBTRy8pZu/jHVUaVaz/NUVussX3ldxSyjTy6qWGxV+gGyXvtpnm+qlY/EFrtoY26km2LoSbX9rdH6Rfl5+LXCFzPNGdV3l4oO8Htc1Wnjr84a7PGUYLSlZJnH4i2YLd9T+WrKFXEcTXX6uTKNNdZ2+aT7MBLWjTuEFfrjTAOzeTw/wiVAaYcqHN6Sx3Pc4Yv70H/g2lOkpMSjmrBdpaZNzetsW/wNOhhvH"))))));  


This evaluates to:  

if(isset($_REQUEST['src'])){  
    show_source(__FILE__);
    die();
}
if(isset($_REQUEST['pass']) && $_REQUEST['pass']=="\x75\x6e\x66\x75\x72\x6c\x69\x6e\x67\x61\x6c\x6d\x6f\x73\x74\x63\x6f\x6d\x70\x6c\x65\x74\x65"){  
    die("Congratulations the flag is: ".file_get_contents('/flag/flag.txt'));
}else{
    die('UnfurlMe.phpEnter the password:

View Source');
}


We can see the password needed is \x75\x6e\x66\x75\x72\x6c\x69\x6e\x67\x61\x6c\x6d\x6f\x73\x74\x63\x6f\x6d\x70\x6c\x65\x74\x65. I pasted this string into the PHP deobfuscator from before to quickly figure out what those characters were. It returned the string unfurlingalmostcomplete.

If you paste this string into the text box original page unfurlme.hacktrinity.me, it shows you the flag: HackTrinity{Functi0nsW1th1nFunc7i0ns}.



PGP

The problem statement is as follows:

I received the attached vaguely insulting email from an anonymous sender. Can you help me track them down and exact revenge?

Flag is HackTrinity{}  


Attached are two files, email.txt:

Hey you. you're stupid.  


and email.txt.asc:

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAABAgAGBQJacv0IAAoJEAXbzVnYRgZCvEgIAL39tKSA7xhj/+O47gYX9E+c  
5pmRyD+RZ5Z2cE5gsC4pQ5OtwoViFw4ANzxNBjSh6ObPRjIIMNrVjOc8MlEMqusl  
SiM7PWwzBg8wbioPmZIIR0gvpLFLOL8VCa9Zn9BlqYC/+aRlSN8OCtP2r4CQ2qN3  
UiAhXDx9sXZbCby/yrzNyRsy18eItYTehD1/aL4tt7EIrisyrtr1k6JISzGjc5ic  
/cwsSLS6esPCM7vn/Tu574hCAyTOGYoTZ41f0ljlPwyQ7Bo0qtl4oOf5aLrvJoyR
ylZQKdU0CHIDIPa5AfrSQ1bEes30wHNUiWkQUZoPnsHoXdJ/+CdCxr2W5Wvrye0=  
=SqZ8
-----END PGP SIGNATURE-----


After some Googling to figure out how PGP signatures work, I came across pretty much exactly what I needed in a security.stackexchange.com post. I can just get gpg to parse the signature file:

% gpg -vv < email.txt.asc 
gpg: armor: BEGIN PGP SIGNATURE  
gpg: armor header: Version: GnuPG v1  
:signature packet: algo 1, keyid 05DBCD59D8460642
        version 4, created 1517485320, md5len 0, sigclass 0x00
        digest algo 2, begin of digest bc 48
        hashed subpkt 2 len 4 (sig created 2018-02-01)
        subpkt 16 len 8 (issuer key ID 05DBCD59D8460642)
        data: [2048 bits]
Detached signature.  


This then gives me the keyid 05DBCD59D8460642, which I can look up:

% gpg --search-keys 05DBCD59D8460642
gpg: searching for "05DBCD59D8460642" from hkp server keys.gnupg.net  
(1)     Eve Heinrichtson 
          2048 bit RSA key D8460642, created: 2018-02-01


HackTrinity{Eve Heinrichtson} is our flag.



BadSSH

Problem statement:

Our sysadmin isn't very good - he doesn't really understand how crypto works, or how to configure OpenSSH properly.

Show him a lesson by finding the privkey.pem and logging in over SSH to retrieve the flag.

ssh -o IdentitiesOnly=yes -o IdentityFile=privkey.pem -p 8022 badssh@145.239.7.7  


This was later appended to the problem statement, after (or perhaps, because) I had already tried many approaches:

Note: You should only interact with port 8022 on this host. All other ports are out of scope.

Hint:  
The sysadmin generated his own RSA key using a custom crazy Python script (he's not a professional cryptographer, so this was probably unwise). The public part of the key is:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbe4wAeBPcaFdSviU1OysvpOMNfdyXZdUwSyTZaB4d4gzny1G5bAVLK5J4M2/nA5nMeh9De/jPjrJiJ6gGIz1uHOAzF2gFcXk4jEBDeFFXd4le3DMyKZ57+WD2rR/D71SCBS44mBumiYSJR7q9tZLi9CTorF+6q19BzAC41bVGJTU0HtsQ10DeoGbghLY6ypKStiXIGYDhFAu0gRvIwjDEGhqPdUWa3ddtg6ml+V5rr40qffeYP2jUxYCjNd/LaF5wUsVOL7ZrWrhOx2BgDM4emIhyyXCHAAjGxFC3yAck+JYBVz8FShXdn/S4liVXyrrsyW6ah32uH6j2yazUJLUP  
Calculate the private part of the key, and use it to login to the box over SSH port 8022.  


This problem took me way too long.

Firstly, I portscanned the target server, hoping for an open webserver or something. I tried an exploit in the version of OpenSSH (7.2p2) this server was running, but this led nowhere. I looked at the public key, took it apart into its modulus and public exponent.

I searched the modulus in FactorDB, googled parts of the key to see if they were leaked somewhere online, did all sorts of recon.

I converted the key to a more useful format early on:

-----BEGIN RSA PUBLIC KEY-----
MIIBCgKCAQEA23uMAHgT3GhXUr4lNTsrL6TjDX3cl2XVMEsk2WgeHeIM58tRuWwF  
SyuSeDNv5wOZzHofQ3v4z46yYieoBiM9bhzgMxdoBXF5OIxAQ3hRV3eJXtwzMime  
e/lg9q0fw+9UggUuOJgbpomEiUe6vbWS4vQk6KxfuqtfQcwAuNW1RiU1NB7bENdA  
3qBm4IS2OsqSkrYlyBmA4RQLtIEbyMIwxBoaj3VFmt3XbYOppflea6+NKn33mD9o  
1MWAozXfy2hecFLFTi+2a1q4TsdgYAzOHpiIcslwhwAIxsRQt8gHJPiWAVc/BUoV  
3Z/0uJYlV8q67Mlumod9rh+o9sms1CS1DwIDAQAB  
-----END RSA PUBLIC KEY-----


I tried putting this into RsaCtfTool, which does most of the legwork in checking for common vulnerabilities and flaws in RSA keys. It crashed after hitting a recursion limit trying to get a GCD, so I added the following after the imports:

import sys  
sys.setrecursionlimit(1500)  


Then it crashed for a dumber reason so I applied the fix, and tried again.

 % python RsaCtfTool.py --publickey real_rsa.key --private
-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEA23uMAHgT3GhXUr4lNTsrL6TjDX3cl2XVMEsk2WgeHeIM58tR  
uWwFSyuSeDNv5wOZzHofQ3v4z46yYieoBiM9bhzgMxdoBXF5OIxAQ3hRV3eJXtwz  
Mimee/lg9q0fw+9UggUuOJgbpomEiUe6vbWS4vQk6KxfuqtfQcwAuNW1RiU1NB7b  
ENdA3qBm4IS2OsqSkrYlyBmA4RQLtIEbyMIwxBoaj3VFmt3XbYOppflea6+NKn33  
mD9o1MWAozXfy2hecFLFTi+2a1q4TsdgYAzOHpiIcslwhwAIxsRQt8gHJPiWAVc/  
BUoV3Z/0uJYlV8q67Mlumod9rh+o9sms1CS1DwIDAQABAoIBAQCVrSJjq3+33nw2  
lC9hptEjs6mXqM6HfM2vGn+tt6BaNq8gX/qCndTaV4OSb1mPzFfVJy8s4V1jhmfG  
Ik8mqF+bORv4G603kRPe6V5l5KZsJLvMshRD0DghSYbDknrILu2NoaTMuYaZwsxc  
pksGVs4t8ds9xVefQIE8qLeRVURC6ZeaQXWHlBQoNt7cWEFtcp4XnlthllWl75no  
pAvwN/GJxREaTdtUflerxCJ5JoxElvlsuJDGO2L/5xy8wowGGe9JnUe9qwxTfrGs  
0wE6Gy4IkR1ffmRxB2gzrl2qOk8eRoLJvETTIYZyyhbAc7W6vrIHSyikSgOlmfll  
3BRAijkhAoGBAO0KBBwqBk+p9ve0LCkjoKb49OySJXy9cHKr+Gc0ZMGQpuiFGIHE  
rpBP+xZghyR5yNHodkKJE9mdccv5vYJEk4Z8ejzJwyaw9ogJhZeH/+sJ3shGZYcR  
XRWgU84kS4x/YMo7Q7RoXSEZZH/t7mNgkPYXA37U1Q71T/Okgr+pPVy/AoGBAO0K  
BBwqBk+p9ve0LCkjoKb49OySJXy9cHKr+Gc0ZMGQpuiFGIHErpBP+xZghyR5yNHo  
dkKJE9mdccv5vYJEk4Z8ejzJwyaw9ogJhZeH/+sJ3shGZYcRXRWgU84kS4x/YMo7  
Q7RoXSEZZH/t7mNgkPYXA37U1Q71T/Okgr+pPKuxAoGBALfleWLQR60uU2fA5DHW  
biCcLIgMqgCoh8CvIjwPLcvuCU+DUov6puNW6ZlmsfHeeapACt97sWa2+z4gNqIF  
yd4gXEUk0r4FtH29xWLWI/mY4rnFw6aSFgFdLSdUiTgq6lB6wgAIp5eyN4H4eWWn  
2U7Sc+fF/rVoI+sFylofVnfJAoGBAM5+nz35zi2wHxmCt8XO57ENyDAe0NFuJnt/  
HJKrreqCHSUKbWL++CN3yYCg7pn0DeHu5Lbpu4UkB3JuSY0mOG48GjDCg2M9Xkb8  
JIjxTRxwKMfHq8KSecjRNrCqJbZrcOI75qtPD3I6MLbRi46/HQmE2uKufjzdr5zM  
f6p/v/7BAoGAVy75wz1UYADMSTl5VB2Ljg35l/lo+ADt1akGunSUUsiE9dg8Cwoy  
BGd5LA4wDnXx84mUeC6zd9FkiA9uqqBVUvCasCE0TvZO54bXETKI8cuQGINOR1DM  
ZKSgGDPbd8jB1ru5Rq+y4Aj46lyw2B9vDPc6E3QweWo+ea0CZcWWznA=  
-----END RSA PRIVATE KEY-----


I didn't expect that to work! But sure enough, putting that private key into private_key.priv and trying to ssh:

 % ssh -o IdentitiesOnly=yes -o IdentityFile=private_key.priv -p 8022 badssh@145.239.7.7
===========================
!Congratulations!

. You solved the challenge, here's your flag:

HackTrinity{fermat_factorisation_is_beautiful}


==========================Connection to 145.239.7.7 closed.


And HackTrinity{fermat_factorisation_is_beautiful} got me the points.

I'm guessing by the flag that the issue with the key was that it used close primes to create the key, allowing a shortcut (ie. fermat factorisation) to cracking it. But I'll have to read up on this later. Thanks for RsaCtfTool for doing that for me!



Have been silently taking part in the https://t.co/CnWBR5wx7L CTF in between meetings and stuff over the past week, had a lot of fun. Just solved the last challenge and got the final flag 🎉

Sásta.

Thanks for organising @ducss / https://t.co/JYMsVG8oMQ / @netsoc pic.twitter.com/XZwRohL5KI
— noah ó donnaile in der schweiz (@iandioch) 2 February 2018



Hashcode 2017
Noah Ó Donnaile — Thu, 04 Jan 2018 14:52:41 GMT
Long ago, I meant to write about this.

I took part in the Hashcode 2017 online round with Cian Ruane. He did a great write-up of our entry!

We came 1st in Ireland, and 164th globally, and had a lot of fun.

Go read his post about it xo


Students in CPSSD Provide 72% of Ireland's Total Score on Kattis
Noah Ó Donnaile — Sat, 15 Apr 2017 03:53:47 GMT
Kattis is an online programming problem archive that gives you points for each problem you solve in relation to how difficult the problem is. It then aggregates the total points of people within a university, and within a country, based on an adjusted sum of the total points of all members of each group.

DCU, the university I attend, is now in the top 50 in the world on Kattis, and Ireland is in the top 30 countries there.

I recently wrote a piece on cpssd.net about how much of Ireland's score on is contributed by myself and my coursemates. It turns out that more than 33% of the score of Ireland comes from my points 🎺

You should read the piece here.

Noah xoxo


30 Day Music Challenge: What music I listen to vs. What I think I do
Noah Ó Donnaile — Sun, 09 Apr 2017 03:11:24 GMT
The following image was floating around the internet this month (in varying degrees of blurriness):



The goal is to share a song on social media for each day of the month of April matching the respecting description. Being the wannabe music fan I am, I was interested in trying this. However, I am sitting on what many (I suppose) are not; with the wonderful last.fm, I have a record of every song I have listened to (with many gaps, granted) over the past few years. I also have the scripting know-how to look through that data.

With this in mind, following is a look at what song lastfm tells me best suits each category, compared to what song I would actually choose when trying to be a music-intellectual to impress my Twitter followers.

Day 1: A song with a colour in the title

The programming for this is easy enough - take a sorted list of my most-played songs, and find the first one that mentions a colour in the title. simple.wikipedia.org keeps a colour list that will suit our needs. I had some old code from making my homepage that dealt with grabbing data from last.fm, so I cannibalised from there and created this repo of scripts for this blog post. This script did the above colour-matching, and spouted the following as my favourite song with a colour in the title:

'Ruby' - Kaiser Chiefs  




A great song! Alas, one that I have not listened to much lately, so is not on my mind. My human-brain answer, to oppose my computer-brain one, would be Rose Rouge by St. Germain. Much higher-brow, and more matching my recent tastes in music.



The script output some other interesting songs, so here are the following ten in order of plays (and the colour they were flagged with, ftb):

'Just Be Good to Green (feat. Lily Allen)' - Professor Green (green)  
'Gold Digger (feat. Jamie Foxx)' - Kanye West (gold)  
'Out of the Blue' - Julian Casablancas (blue)  
'City Of Gold' - Professor Green (gold)  
'Black & Gold' - Sam Sparro (black)  
'Gold on the Ceiling' - The Black Keys (gold)  
'Black horse and the Cherry Tree' - [unknown] (black)  
'Rose Rouge' - St. Germain (rose)  
'Black Plant' - The Last Shadow Puppets (black)  
'Big Yellow Taxi (feat. Vanessa Carlton)' - Counting Crows (yellow)  


Before I fixed it to stop recognising partial words as colours (eg. "I Predict A Riot" was flagged as having the colour "red" in it), it also output Greenback Boogie, by Ima Robot. I... I really need to listen to that song again. I guess it counts as having a colour in its title too?



I was worried that Come Out Ye Black and Tans would show up here, but probably for the best, I don't seem to have scrobbled it much.

Day 2: A song with a number in the title

The script to magic up an answer for this is very similar to the last one, but with some numbers instead of colours. This list of number-words was hand-compiled. The top result was this:

'Four to the Floor' - Starsailor  




Nice! I hadn't thought of that one. The song in my head was Two Minds by Nero, one of my most-played artists of all-time.



It appeared very far down the list of other outputs, of which here are the next ten:

'Miami 2 Ibiza' - Tinie Tempah (2)  
'Daft Punk - One More Time' - [unknown] (one)  
'Seven Nation Army' - The White Stripes (seven)  
'One Day Like This' - Elbow (one)  
'Season 2 Episode 3' - Glass Animals (2)  
'Ten Cent Pistol' - The Black Keys (ten)  
'No One Knows' - Queens of the Stone Age (one)  
'One (Your Name) [feat. Pharrell]' - Swedish House Mafia (one)  
'Mozart Piano Concerto, K. 488 (Zoltán Kocsis)  movement 3' - [unknown] (3)  
'Song 2 (2012 Remastered Version)' - Blur (2)  


They're good tunes, Bront. 

Because I have listened to different cuts of Song 2, it has multiple lower-playcount listings, but should maybe be higher? This is the case with a lot of songs undoubtedly. You can also see in the list above that wherever I was playing One More Time by Daft Punk had it mistagged :(

Day 3: A song that reminds you of summertime

This is a harder task to do programmatically. I decided to take a similar approach to the previous task, and compile a list of words I think of when I think of summer - things like "holiday", and weather-related words. It's not perfect - it misses "summery-feeling" songs (like this one, for example, that feels sort of swishy and summery to me, but mentions none of those themes).

The script gave the following results, among others:

'When the Sun Goes Down' - Arctic Monkeys (sun)  
'Sunny Afternoon' - The Kinks (sunny)  
'Staring At The Sun' - Simple Kid (sun)  
'Honey Sun' - Elbow (sun)  
'Crying In The Rain (Funkerman Remix)' - Guru Josh Project (rain)  




My own choice was Sunny Afternoon by The Kinks, which is the second output here. Is that a success then?



Day 4: A song that reminds you of someone you would rather forget about.

I have no idea how to do this in script form. Perhaps I could look at what songs I listened to intensely during some period in the past, but suddenly stopped listening to?

In any case, my organic answer is Bohemian Like You by Dandy Warhols. That one is still a favourite, tainted though it may be.



Day 5: A song that needs to be played loud

I wasn't quite sure how to figure this out. I came to the conclusion that the loudness of my music is strongly correlated to the speed of it - fast, exciting songs are what make me turn up the dial. As such, I queried the beats-per-minute of my top 250 most-played songs using the getsongbpm.com api, that they kindly provide freely. Here is the python-fu to do that. There was a lot of messy data, I only got values for 101 of the 250 songs, and I have no idea how correct the numbers are, but c'est la vie. Here were the quickest 10, supposedly, as per this script:

'Hotel Yorba' - The White Stripes (196 bpm)  
'From the Ritz to the Rubble' - Arctic Monkeys (190 bpm)  
'Arabella' - Arctic Monkeys (180 bpm)  
'End Credits' - Chase & Status (176 bpm)  
'Fool Yourself' - Chase & Status (176 bpm)  
'Clint Eastwood' - Gorillaz (168 bpm)  
'Brianstorm' - Arctic Monkeys (166 bpm)  
'Everyday I Love You Less and Less' - Kaiser Chiefs (160 bpm)  
'I Predict a Riot' - Kaiser Chiefs (158 bpm)  
'Never Miss a Beat' - Kaiser Chiefs (156 bpm)  




I'm interested in this result. These are fine songs, but not ones that jump into my head when I think "loud"... Fell in Love with a Girl by The White Stripes is what I would have chosen. It has been my ringtone for many years, and I jam to it that time my phone rings every few months. The script-suggested one was also by The White Stripes, so it mustn't be too far wrong?



I wish that song was longer.

Postmortem

That was interesting! I've been reminded of some good songs, and I always enjoy playing with data, so it was a win-win. The programmatic answers seem to be skewed by my tastes a year ago; there's an abundance of rock in there, while lately I've been into a wider variety of genres. That was expected.

I look forward to continuing this exercise in the future, but it takes a while, so I'll have to stop here.

Feelin' so bohemian like you,

Noah

(The picture accompanying this post is a screenshot of some lads a-jammin' in the Bohemian Like You music video)


123rum123 at the IrlCPC 2017
Noah Ó Donnaile — Mon, 03 Apr 2017 20:37:51 GMT
The Irish Collegiate Programming Contest (IrlCPC) 2017 was held on Saturday, 25th of March, in sunny University College Cork.

I was competing with two of my housemates, Cian Ruane, and Ross O'Sullivan. Myself, Cian Ruane, and Stefan Kennedy came 2nd place at the IrlCPC last year, so I fancied our chances this year.

Our team, 123rum123 (named for the password of our other housemate's boyfriend), flew to the top of the board from the beginning, finishing first in the practice round.

We were hyped for the real competition, and quick off the block, we solved 5 of the 6 problems with less stress than we expected. We worked very well as a team, and had a lot of fun! 

However, that last, forsaken problem... It was a topological sort, and my only previous problem I could compare to it was a Project Euler problem I solved. However, I couldn't find a way to apply any solutions I knew for that problem to this, and any graph-based or set-based solution was too slow.

Alas, we only got 4 points for problem 5. It was the only we didn't get full marks in, leaving us 6 points from a perfect score.

While we had solved the other problems faster (we think), 2 teams from Trinity College passed us out (fair deuce to them) and solved the final problem, leaving us in 3rd place at the end of the competition.

We were (of course) delighted - third isn't bad at all :) We are somewhat familiar with the team that came 1st, and we beat them in the recent Hashcode, so it wasn't all bad!

Thanks to Google, Newsweaver, and UCC for the good day that was in it. We'll win in future xo


Hack the Dot Dublin
Noah Ó Donnaile — Sun, 26 Feb 2017 03:03:00 GMT
On Wednesday the 22nd of February, I took to the southside to take part in Hack the Dot, a domain-name-driven hackathon organised and hosted by Rightside.

I am a big domain name watcher, and a vocal fan of the new generic TLDs. Rightside are a Donuts competitor, and I didn't realise that they owned the TLDs of two of my stellar domains, blurry.video and meme.rehab (I want to host a GNU Social instance at the latter, when I get time). It was intiguing to peek behind the curtain, even if just for an evening! Competitors in Hack the Dot were forewarned of a free domain, along with a few beers, so I was excited.

It was a lot of fun! At the beginning of the competition, the teams are all presented with a domain name, the same for all teams. You then have 90 minutes to come up with a business based on that domain. The inspiration-tickler we were given was hack.live. 

I was in cahoots with 3 others, all from UCD. The idea we pitched (to the CEO of Rightside, via GHangouts) was a distraction-killing browser extension that resurfaced old embarrassing Facebook posts if you spent too long on blacklisted websites. If you ignored the warnings, it proceeded to message random friends telling you to get back to work. Rather than hacking anything physical, we aimed to envision a lifehack, and this one sounded fun.

We didn't win, but I was surprised with a free tshirt and a good evening of brainjamming. There was a code distributed to get a free .live domain with name.com, the registrar front of Rightside. Alas, I was too busy working in the following 24 hours to redeem the code in time.

Was at the Hack the Dot with @rightside in Dub yesterday eve - was a good bit of craic, fair play (y)
— cash for ash ketchum (@iandioch) February 23, 2017  



I had a lot of fun, and met a few new people, so I'd definitely recommend an event like this. However, it didn't really get me going like a longer product-focused hackathon, where there is the time (and expectation) to build a working prototype. The previous hackathon I was at, HackYourWayDay, had a similar problem, but there was more time there to put a prototype together.

It's nice to forget about proper work for a few hours anyway, and have a bit of craic. I'll keep my ear to the ground for similar happenings over the next few months.

Cheers to Hakeem, whose name was on the ticket and who I was clumsily masquerading as for the first 3 minutes of Hack the Dot.


Coireacht - Our 3-Hour-ish Hackathon Project at HackYourWayDay
Noah Ó Donnaile — Sun, 12 Feb 2017 06:13:12 GMT
On Friday the 10th of February, myself and my compadre Cian Ruane took part in the HackYourWayDay hackathon in the NDRC that we had seen publicised around the place the previous week. The hackathon was themed around Eircodes, which are Ireland's new controversial postcodes. I'm interested in data, mapping, and hacking, so I was pretty interested. Unfortunately, I was late to the safari because I had a hospital appointment that morning, but I was there for the fun bits: pitching, building, and presenting our product, Coireacht. Naming things is said to be one of the most difficult parts of tech, but I have a cheat: use the Irish word for whatever I'm looking at. Coireacht is the Irish for crime. 

Exploring government data at today's hackathon #HackYourWayDay pic.twitter.com/pOdJ0oLqm0
— data.gov.ie (@GovDataIE) February 10, 2017  



Above is a tweet with a nice pic of, from left to right: Cian, myself, and blurry Adrian, the CTO at Opening.io. Opening was the company where I was lucky enough to have my first job, enter my love affair with machine learning, and even have drinks on a boat with; it also now happens to be based in the building the hackathon was hosted in. On the board in the background of the picture you can also squint at our tight time limits for the hacking.



In advance of the hackathon, some links to location-related datasets were shared with us. It rubbed me the wrong way how many of the datasets were for the chrome and crime of Dublin City, and how few were countrywide. At the hackathon, the vast majority of people in the room were Dubs or near-Dubs. Three quarters of the people in this country are not Dubliners, and I wanted to put together a project that helped them. My teammate Cian is also a countryboi, and either had the same thoughts, or just didn't mind abiding me. 

The idea we came up with was a tool to help students from the countryside, or people about to immigrate, find out more about potential accommodation when they are moving for college or work. Before university when I was seeking accommodation in Dublin, came across a potential place, and mentioned the price and address to my mother (who lived in Dublin's fine city once upon a pale moon), she would share a lot of unsolicited information on whether the price was fair, or if it was a dangerous area. Had she not known these things, I would have been clueless. Her information was also decades old. A tool to accessibly provide modern information like this may help someone. We had pinpointed our problem; following is our enthralling technical solution.



Here is what we set out to do:


Take an eircode from the user in a nice and friendly way  
Find crime data for the location  
Return a rating for how dangerous it is  
Find recent rental prices for nearby properties  
Tell the user how much to expect to pay in rent here


We achieved as far as #3 above, tickling the edge of #4, over the three-ish hours of the hackathon. We opted to whip up a webapp of sorts, that you can try out live right here, and eyeball our code here.

You can git-blame the repo to see who did what between me and my navigator Messr Ruane. I have a fair amount of experience with Flask; it is perhaps my favourite library or framework of all time, making sometimes messy webdev as pleasant as a trip to the opera. Cian is not so familiar with Flask (or the web, the old fart), so I threw together a barebones webapp, and shipped it off to him. He bound together APIs to turn an eircode into an address, and from there into a latitude and longitude.

We found an open dataset of the crimes within the remit of each Garda station in the country over the past few years (although I am still seeking the exact licence for this data). It was laid out like this:

  
  
ID Garda Station Division X Y Number of Occurances of Crime P in Year Q  
  
  
20441 Abbeyfeale Limerick Division 112219 126928 insert grizzly crime number here  
  


I am not sure what the context of the X and Y was - perhaps an offset to the numbers mentioned for the range in the metadata of this dataset. Instead of fiddling with offsets, I doodled a script that took the name of each Garda station (eg. Abbeyfeale), cheekily appended Garda Station, Ireland to it (to make Abbeyfeale Garda Station, Ireland, and queried Google maps for an exact location, then output all this in a CSV. It worked for all but 8 or 9 stations, which we discarded in the name of time constraints.

We only took the numbers of violent crimes and thefts into account; we don't think that tax fraud makes an area more dangerous. In fact, tax fraud might be negatively correlated with an area's safety...

Cian wrote a wee bitta code that took the coordinates of a location, and the coordinates of each Garda station, found the nearest one, and used it to figure out a crime rating for the property. We pretended the world was flat and used the latitude and longitude as points on a 2D plane to calculate distances, again because of time constraints.



A screenshot of the homepage, which is just a light description of the tool, a box for you to type an eircode into, and a big "go" button.

I wrote a frontend! I used Tachyons, a CSS toolkit I've used and enjoyed before that makes it a bit less painful to get things looking good (on both desktop and mobile!), along with Jinja2 templating to put together the pages in a nice, modular way. I used the vital tool of StackOverflow to help me put a map onto the page, in order to visualise where the Eircode was in the real world. I also used some messy Jinja-ing and CSS-ing to fill 5 stars based on the crime rating of the area, and highlight it red for bad, orange for okay, and green for good.



A screenshot of the results page for an eircode, with the address, coordinates of it, a score out of 5 stars for how safe the area is, and an OpenStreetMap embed showing where the place is.

Myself and Cian presented our project to a panel of Dubliners (who didn't relate to our original problem perhaps!). We didn't win any of the 3 prizes, so no 5000€ for us. I was very proud of what we managed to put together however. We were one of only 2 teams that had a live interactive demo, I think, and the only team to have a tool working on mobile. After all of the prizegiving, one of the judges came over to us and told us we were *this close* (pinches fingers) to the top dogs. Throughout the presentations, they pushed the point of how close the competition was. I think they always do that, but it felt nice to know we were maybe almost in the money.

The workshopping has begun! Ideas forming #hackyourwayday @Version1Tweets pic.twitter.com/nqPOWVBl9A
— Enda Diggins (@endadiggins) February 10, 2017

A tweet with a picture of us looking for help with some data. I don't remember it being taken, but I'm happy enough because for some reason I look okay in it. I am a little suspicious.



I had a lot of fun on the day, and I think Cian did too - maybe he'll be kind enough to write up how it felt from his point de vue. If we get the time and conjure up some motivation out of our rear ends, Cian and I may augment the project a little bit at some stage. I wasn't expecting the free pizza and beer that appeared in the evening of the event, it was a nice way to finish! I'll ignore the fact that I followed it with more drinks. 

This was my first time playing with government data, and it was fun - I look forward to doing it again in the future. I hope someone finds some use in our tool. If you're interested, you can play with it here!

Bonus points to me for peppering this piece with the names of 8 different web browsers. Try and catch them all!


Parsing Project Euler Progress with PyOCR
Noah Ó Donnaile — Sun, 15 Jan 2017 02:14:27 GMT
I do a lot of problems on Project Euler. It is a really cool site full of mathematically-flavoured programming problems that are too difficult for me. My homepage right now keeps some live stats about my progress on Project Euler and some similar sites. Unfortunately, Project Euler doesn't provide any sort of public machine-readable data on how many problems you have solved. However, it does provide an image badge like the following:



I want to use OCR (optical character recognition) to parse the number of problems I have solved out of the above image, and allow it to be displayed on my homepage.

I'm going to use Python3 for this, because Python is easy, and speed isn't critical for this application (it will probably be run as a daily cronjob or something). 


  The Setup


There are some nice articles out there that suggest tesseract-ocr is my best bet for some digit-recognition. pyocr is the most actively maintained wrapper for it that I can find as of early 2017. Installing this with pip is trivial, although I had to play with Pillow installation first on my Ubuntu 14.04 VPS.


  The Heist


Let's get coding. You can see all the work in the euler-foiler github repo (the repo name rhymes if you pronounce Euler correctly xo). There is not much to it. Here is the guts of the parse function in parse.py:

def parse(image_loc):  
    tools = pyocr.get_available_tools()
    if len(tools) == 0:
        return "Error: No OCR tool found"

    # should be 'Tesseract (sh)'
    tool = tools[0]

    orig_image = Image.open(image_loc)

    # crop to only the section with the number of problems solved
    cropped_image = orig_image.crop((47, 40, 97, 60))

    # double the size of the image so the OCR has more to go on
    resized_image = cropped_image.resize((100, 40), Image.ANTIALIAS)

    digits = tool.image_to_string(
        resized_image,
        builder=pyocr.tesseract.DigitBuilder())

    return digits


Most of the code is taken directly from the README of the pyocr project.  We explicitly declare that the script uses the Tesseract (sh) tool, as it provides us with a model to only recognise digits.

We use a nice Pillow method to crop the Project Euler badge to only give us the part we care about (the number of problems solved). The hard-coded parameters are the left, top, right and bottom of the bounding box, and were figured out in the GIMP.

If we plug this into tesseract straight away, then it will not give us the right answer, as it is looking at a very small image (50x20 pixels). We therefore use Pillow to double the size of the cropped image.

We can then pop this into tesseract, specifying the digit-recognising model. We do not have to specify a language, as whatever the system default is will do for digits.


  The Getaway


We're done! That wasn't nearly as complicated as I expected. The most difficult part was installing the dependencies.

If you look at the full file, you will see some extra bits to make sure the path the user gives is valid, etc. There is also a very small test script in the repo.

To use this script, all you need to do is grab your badge (eg. wget https://projecteuler.net/profiles/iandioch.png), run the script with the right path (python3 parse.py iandioch.png), and you're away!

It accurately recognises my 128 problems solved right now. I look forward to it recognising higher numbers in future. I will work now to incorporate the data into my homepage, and now that I see OCR isn't so scary when I use nice libraries, I can apply it to a few other problems...


dia-log, a diabetes management system
Noah Ó Donnaile — Wed, 21 Dec 2016 01:26:06 GMT
Context

I have type 1 diabetes. It's not a big deal, there are much worse afflictions that can hang on a person. Diabetes just means the occasional pain in the arse (sometimes literally xo).

I am supposed to get up early, eat 3 steady solid meals, and exercise; as is the case with every university student, none of the above happens more than one day a week. I am also discouraged from excessive alcohol, but that is another scéal. Every once in a blue moon (between once a month and once  every 18 months) I have to go to the hospital to see a nurse or doctor or dietitian, and look at the trends in my blood sugar. 

The biggest hurdle with diabetes is tracking my blood sugars. I use a glucometer like this to measure my blood glucose level. This is a device with a pathetically small battery and a sufficient amount of internal memory, that stores a history of my blood sugar readings in mmol/L, next to a timestamp of when that reading was taken.

It is a closed system; I cannot access that data. I have dreamed of writing ML tools to predict a hypo, suggest that I should take more insulin at certain meals, or even just graph my sugars. It is fine (but against my libre ideals *sob*) that it is a closed system, I can get past that. But every time I visit the hospital, I am supposed to bring all of my blood sugar data from the previous weeks or months in written form, so the doctor does not have to squint at the tiny screen on the glucometer and scroll through readings one at a time. Ugggghhhh.

I have for years been planning to sit down and write a system to digitise the process of logging blood sugars in an open and friendly format, and this afternoon I finally did it.

Project

Presenting: dia-log (and its accompanying frontend);

Dia-log (diabetes logging, perhaps the most boring project name I have ever envisioned) is a small backend+frontend to allow a number of users (or a single one, as in my case) to upload their blood sugar data to the cloud (at least, a VPS in my case). 

Right now, it is a lightweight system.

The backend is a JSON-based RESTful server implemented with Flask (my favourite framework of all time, it's so nice to use). So far there are only two endpoints; one to add a new blood sugar reading, and one to add a new user.

That's right, it's a multi-user system, with some small authentification of those users, based on a username and (bcrypt+salted) password.

The frontend is as yet a simple static form (with a nice date picker based on Pikaday and moment.js) that sends things to the back end, built with a python&jinja-based static site generation script. I hope to spice it up with some CSS (TACHYONS is very nice) at some stage, and add some d3-flavoured visualisations; but baby steps.

Design Decisions

I was asked why I separated the project into two separate repositories. It seemed to make more sense;


The two parts are, from a programming point of view, completely separate. They communicate over a RESTful API but share no code.
I would like to create alternate clients at some stage (eg. a mobile app), or at least leave the option open for it.
The server is a dynamic app but the frontend is a static site better served with nginx or apache than Flask for performance and configuration reasons. Putting them in the same repo would conflate and confuse this.


I have configured my nginx to make the frontend available at dialog.blurry.video, and the backend at dialog.blurry.video/api, with some rewrite-fu in the middle to make Flask think it is at a subdomain and not /api. 

I wanted dia-log to be very configurable, and if you look at the config.json that comes with each repo, I think you'll see I succeeded reasonably well with that. 

All of the blood sugar readings are put into plain old textfiles (of the path //), so a backup is as simple as a cp (or in my case, a git push). I have read lots of praise for plaintext data storage systems, and with how simple this data is, a database would likely be overkill. 

I need to document, and add a few more features, configuration options (I ignore timezone woes, and there are other measurement systems than mmol/L, to name a couple) and tests, but who knows; maybe another sorrowful diabetic will find these tools useful someday?


RIP CPSSD
Noah Ó Donnaile — Mon, 12 Dec 2016 14:13:52 GMT
I am doing a BSc. in Computational Problem Solving and Software Development (CPSSD) at Dublin City University; DC122 in CAO talk. It is a small, specialised course for students who already have some programming knowledge. You can learn more about it at the website I set up for it.

But alas, CPSSD is no more. 

Never again will people have to stumble through its long name. It has also been called PSSD, CPSTD, and PSTD, and a whole host of other letter combinations my mother came up with on the spot. 

Never again will there be bright-eyed LC students throwing together portfolios or stuttering through interviews to get in.

Never more will DCU teams top the charts at everyone's favourite programming competitions. Well, we'll still be around for a couple of years yet ;)

My uni course was cancelled today. I can finish, but no new first years :(
— snoah 🌌❄ (@iandioch) November 17, 2016  



It turns out that hosting a degree with a high barrier to entry is not economically viable, as there were too few students in each year to let DCU turn a profit. 

We were told in quite a terse email that those already in the course can see it through, and finish the degree. However, applications for new students have closed for good.

The funeral for my course after it was cancelled is a very emotional affair #ripcpssd pic.twitter.com/TPRb9fO5Ny
— snoah 🌌❄ (@iandioch) November 27, 2016  



I firmly believe that CPSSD was the best IT degree in the country, and filled a well-needed hole in the education system here. You can see, looking at the current students (sin a rá "me and my boyz") with their wealth of enthusiasm about tech, that they would get bored and frustrated in a normal degree. None of Ireland's tech degree offerings are aimed at people who already have experience, or who are independent learners. None of them suit someone who is ahead of the curve as well as CPSSD.

It makes sense that DCU cancelled the degree, from a financial point of view. But it still makes me sigh.


It's a Ghost!
Noah Ó Donnaile — Mon, 12 Dec 2016 13:47:06 GMT
Bhí sé ar m'intinn an blag seo a athrú ar feadh cúpla mhí. Bhuel, tá sé déanta anois. Táim ag baint úsáid as Ghost anois - tá sé deas go leor, simplí, agus FLOSSy.

Bhog mé an cuid is mó de na postálacha ón leagan sine anseo. D'fhág mé cúpla cinn níos leadránaí sa void ag github.

Chaith mé an tsamhradh ag Google Zurich. Bhí sé dochreidte. Níos mó ná sin, chaith mé mí ar fad ag teasteal. Tá súil agam go mbeidh an tsamhradh seo chugainn comh iontach.

Beidh postálacha nua ag teacht am éigin.

😘

Noah

ID	Garda Station	Division	X	Y	Number of Occurances of Crime P in Year Q
20441	Abbeyfeale	Limerick Division	112219	126928	insert grizzly crime number here