When your data becomes massive and data analysts are eager to construct complex models it might be a good time to boost processing power by using clusters in the cloud … and let their geek flag fly.
Therefore, we use AWS Elastic Map Reduce (EMR) which lets you easily create clusters with Spark installed. Spark is a distributed processing framework for executing calculations in parallel.
Our data analysts undertake analyses and machine learning tasks using Python 3 (with libraries such as pandas, scikit-learn, etc.,) on Jupyter notebooks. To enable our data analysts to create clusters on demand and not completely change their programming routines we choose Jupyter Notebook with PySpark (Spark Python API) on top of EMR.
We mostly followed the example of Tom Zeng in the AWS Big Data Blog post.
For security reasons we run the Spark cluster inside a private subnet of a VPC, and to connect to the cluster we use a bastion host with SSH tunnelling, so all the traffic between browser and cluster is encrypted.
At Babbel our learning content is maintained and created using a custom-made content authoring system based on Angular 1.x. The application has become quite complex by now, counting about 80 custom directives, 35 services and 10 filters, which are tested by about 1400 unit tests (just so you get an idea of the sheer size). The application is under continuous development for two years now and recently we have experienced some serious performance issues for the first time. Our most complex view consists of a spreadsheet-like layout containing, on average, about 50-80 of rows filled with content. When users switched between content packages, resulting in the view being updated, there was a noticeable lag of 2-3 seconds, which became annoying quickly.
This article will walk you through our process and explain in detail how we cut the rendering time in half.
Do you want to learn how to create you own Android app, but you don’t have any developing experience? What about joining your local Android Study Jam?
A study Jam is a community-run study group. Members of local Google Developer Groups support beginners learn Android development.
And this week we hosted in our offices the first session of the “WTM Berlin Android study Jam” season 3!
At Babbel, we revised our test automation strategy about 1.5 years ago. Since then, our focus has been on frontend testing (browser and mobile) for crucial parts of our business. Which, in our case, is mostly that a user can register or login and navigate through the language lessons after subscribing.
I’ll give a short introduction to test automation here, so if you’re a test automation engineer, what I’ll be covering is probably not new to you. If, however, you’re new to the topic, read on!
Jana Rekittke from the Engineering gave us a recap of the EWIT16* conference she attended, presenting some topics and ideas. Enjoy reading the summary of her talk!
Tech workplaces tend to be often typically male dominated. Women might find a “Brogrammer” culture in which they may feel out of place, uncomfortable, or less valued. An unappealing workplace could also be a reason for women to not consider it in the first place. Yet, changes in the workplace tend to only happen when more women start working there.
So the question is: How do we achieve gender parity? And do we actually want that? Does it make sense for a company? Quite probably so: a mixed workplace (think of “not only white middle aged male”) will give the company access to a more diverse pool of ideas. If everyone who makes decisions has the same background, similar ideas and opinions, how can a company/country/… evolve and adapt to ever-changing requirements?
How about women in leadership? Do we need more women in middle/upper management and on company boards? Several studies (see links below) indicate that companies with women in upper management and on boards are more successful than their male-run counterparts by achieving greater productivity, creativity, and profitability. However, companies founded by women receive less funding. Which is odd considering that a company run by a woman is more likely to succeed than a company run by a man.
Sadly, the percentage of companies that have women in upper management is around 70% (big companies, first world), and the percentage of women vs. men in management positions of these companies lies between 0 and 30%. The percentage of women in tech teams is just as low.
We are therefore far from parity - and those who say that feminism or gender studies are outdated and not needed should reconsider their position. There is obviously still work to be done.
For further reading on this broad and interesting topic, please consider these:
As your project grows you might have found the need to split it into several modules. This becomes even more prominent when you’re working in a company where several teams develop the same app, but different features.
It’s not so uncommon to have the same library dependencies accross the modules. Usually modules tend to use the same library to achieve similar things and keep the number of used libraries to a minimum. However, it’s also not so uncommon to accidentally use different versions of the same library in this setup.
Here at Babbel we recently faced this issue and in this blog post I’ll share with you a possible solution to overcome this.
Lots of ideas, people, Club Mate and pizzas. Hello again, with our own review after a successful Hackday.
The 6th Hackday was held at Magazine in der Heeresbäckerei, an impressive industrial monument with a prominent appearance in Kreuzberg, directly next to the Spree. It is extremely important to have a amazing location for such a long, intensive day and we’re happy that we could spend it in this wonderful venue! To make sure nobody was hungry we had lunch from Knofi and delicious Italian pizzas from our good friends at LaPausa.
Unlike the participants of the hackday, our organization team had a little longer than 10 hours to prepare their ‘product’: the first steps towards the event were taken already a couple of months before. It is very satisfying to see how all the small parts we’ve been working on are coming together like puzzle pieces, creating one big event that so many people can enjoy.
Enough background, let’s talk about the actual Hackday!
The 6th Babbel Hackday started at 8 o’clock in the morning - which is considered the middle of the night for some of the Babbellonians - with coffee and sleepy faces, and people happily putting on their work uniforms: this year’s Babbel Hackday T-shirts. With every zip of hot beverage, more and more smiles appeared and our day with ten hours of hacking was ready to begin. The energy levels were rising by the minute and if this Hackday would have had background music, it would definitely be Reel 2 Real’s ‘I like to move it’!