https://www.zeip.fi/en/blogi/ Jyri-Petteri Paloposki https://www.zeip.fi/en/blogi/2021-06-11_trust-and-openness-our-decision-making/ https://www.zeip.fi/en/blogi/2021-06-11_trust-and-openness-our-decision-making/ Fri, 11 Jun 2021 16:00:00 +0300 Turun Sanomat reported a few days ago that Turku will finally publish it's purchasing data next spring – but only for this autumn. I'm glad this is finally happening – last autumn I reiterated in a post how far behind Turku is compared to other communities. Only the data for this autumn will be published, so actually the people will only gain access to a complete year's data on where our common tax money is spent in spring 2023, two years from now. https://www.zeip.fi/en/blogi/2020-10-12_turku-be-forerunner-open-data/ https://www.zeip.fi/en/blogi/2020-10-12_turku-be-forerunner-open-data/ Mon, 12 Oct 2020 21:10:28 +0300 Would you like to know what your tax euros are used for? Should data produced with public funding be usable and refinable by the citizens? Wouldn't it be awesome if a company in Turku could develop profitable and tax-gaining business by inventing a new way to utilise data? https://www.zeip.fi/en/blogi/2020-09-02_im-running/ https://www.zeip.fi/en/blogi/2020-09-02_im-running/ Wed, 02 Sep 2020 23:43:00 +0300 My term in the board of the national scouts organisation ends at the end of this year, and it's time to yield for some fresh ideas. But what to do next? Last night it was confirmed that next spring I'll be working on my election campaign – I was confirmed as one of the Green party candidates for city council in Turku. https://www.zeip.fi/en/blogi/2019-08-09_installing-modsecurity-nginx-debian-10/ https://www.zeip.fi/en/blogi/2019-08-09_installing-modsecurity-nginx-debian-10/ Fri, 09 Aug 2019 12:19:12 +0300 Debian finally has a ModSecurity package ready for use in the repository. However, using it with Nginx requires a few steps, which it took a while to figure out. # Create a work dir for the compilation process $ mkdir modsecurity-tmp # Install modsecurity $ libmodsecurity3 libmodsecurity-dev # Fetch sources $ git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git $ apt-get source nginx-full # Figure out the necessary configuration options $ /usr/sbin/nginx -V $ cd nginx-[version] # Configure. Replace NGINX OPTIONS with all the options from the above command except for options starting "--add-dynamic-module". # https://github.com/SpiderLabs/ModSecurity-nginx/issues/159 and https://github.com/SpiderLabs/ModSecurity-nginx/issues/117 $ ./configure [NGINX OPTIONS] --add-dynamic-module=../ModSecurity-nginx # Compile. $ make modules # Copy to location $ sudo mkdir /etc/nginx/modules $ sudo cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules/ngx_http_modsecurity_module.so # Load the module. $ echo "load_module /etc/nginx/modules/ngx_http_modsecurity_module.so;" | sudo tee /etc/nginx/modules-available/modsecurity.conf $ sudo ln -s /etc/nginx/modules-available/modsecurity.conf /etc/nginx/modules-enabled/ # Add configuration $ sudo mkdir /etc/nginx/modsec $ sudo curl -o /etc/nginx/modsec/modsecurity.conf https://raw.githubusercontent.com/SpiderLabs/ModSecurity/v3/master/modsecurity.conf-recommended # You might also want to change the SecRuleEngine parameter in /etc/nginx/modsec/modsecurity.conf to On. # Load the missing file per https://github.com/SpiderLabs/ModSecurity/issues/1941 $ sudo curl -o /etc/nginx/modsec/unicode.mapping https://raw.githubusercontent.com/SpiderLabs/ModSecurity/49495f1925a14f74f93cb0ef01172e5abc3e4c55/unicode.mapping $ echo "Include \"/etc/nginx/modsec/modsecurity.conf\"\nSecRule ARGS:testparam \"@contains test\" \"id:1234,deny,status:403\"" | sudo tee /etc/nginx/modsec/main.conf # Then add these lines to your site's server block: modsecurity on; modsecurity_rules_file /etc/nginx/modsec/main.conf; # Restart Nginx $ sudo service nginx restart Voilá. If you changed SecRuleEngine to on, adding ?testparam=test to your website URL should return Forbidden 403. Now you can start figuring out the correct ruleset for your site. https://www.zeip.fi/en/blogi/2019-08-08_fixing-hacked-drupal-site/ https://www.zeip.fi/en/blogi/2019-08-08_fixing-hacked-drupal-site/ Thu, 08 Aug 2019 17:20:41 +0300 Figure out what's going on Try to figure out what exactly has been done on the site. If there are no visible content changes, it is possible that your site is used as a proxy server for black-hat SEO. Check the web server access logs to see if there are any URLs that shouldn’t be there. Checking Google Search console might also give you some insight in this. Fixing the site It is always best to restore the site from a clean backup. If this is not possible due to circumstances, make sure to clean the site properly using the instructions below. Please note, that cleaning a hacked Drupal site requires advanced knowledge of Drupal development! If you’re uncertain on any of the themes mentioned below, it’s best to get help from a seasoned expert. Server config Make sure you have a valid firewall preventing access to database, cache service etc. If at all possible, consider adding a WAF (modsecurity). System files Check if the www user of the hacked site has access to other directories than the webroot. Any files the user has read access to should be considered leaked, and any files the user has write access to compromised.<p> # Run as the WWW user to discover any writeable files. $ find / -perm -2 -type f -exec ls -la {} 2>/dev/null \; Rebuilding Fetch all code from upstream, for example using drush make or composer. Check any custom code and themes for backdoors! For Drupal 7 I recommend Wunder’s build management system, which makes deployments and updates easy by clearly defining all versions and patches necessary for composing the site. Modules Check the system table in your database. It is possible to reference module files outside the Drupal root – so a backdoor module could reside for example under /tmp. Be especially wary of filenames that start with dots (for example ../../tmp/…) and filenames ending in something else than .module – for example .jpg or other innocent-looking filenames.<p> When reviewing custom code in modules, themes and database PHP fields note that often the hacker has made an effort to conceal the inner workings of the code to prevent it from showing up on greps etc. This means that function calls may be hidden by concatenating the function name in a variable (for example $cmd = "st"."r"."_"."rep"."la"."ce";. Code can often be hidden as Base64 encoded and decoded on runtime. This highlights the fact that all code must be manually checked, not just grepped for suspicious content! Any PHP in the database The core PHP module shouldn’t be used, but if it has, make sure that there are no backdoors in the PHP code saved in the database – for example block visibility definitions and block or node contents. Make sure to search for all opening tags – both <?php and <?= work in most configurations, but there are also other possibilities depending on the PHP version and configuration.<p> I also wrote a Drush PHP script which helps find the PHP code in a Drupal installation for audit. Note that the script doesn’t cover all modules, so be sure to check the coverage and make other checks as necessary!<p> When checking the PHP code in database, refer to the checking instructions above in modules section. Users and permissions Check the site users and remove any user accounts added by the attacker. Additionally make sure to check your user permissions – be especially wary of the permissions assigned to unauthenticated users (and authenticated users, if you have more than a few trusted users.) Change the keys Make sure to change: Drupal secret in settings.php to prevent using one-time login links, which can be easily forged if the attacker knows the drupal secret and the user's password hash. Any SSH keys that are readable to the WWW user Database keys If your site uses any APIs (for example mail sending or data retrieval), change the access keys or passwords! All user's passwords. Clear the sessions Clear the sessions database table to disable any sessions the attacker has opened, otherwise they might be able to get in using an old session. Cron and running processes Check the crontab of the WWW user for crons that attempt to re-install any backdoors.<p> Also check all the processes the WWW user is running and kill any unknown processes. Disallow any proxied URLs from your site If your site had either unwanted content or proxy URLs, it is useful to speed removing them from the search engines. <p> First make the unwanted URLs respond with the HTTP status code 410 instead of the standard 404 in your WWW server – for example using something like this in Nginx: location ~ ^/.shop/ { return 410; } You should also add the URLs to your robots.txt as disallowed with a wildcard, if possible. These actions together should speed up removing them from the search index. If there are only a few URLs, you might of course request removing them manually through the search engines’ control panels. Monitoring the situation It’s important to keep monitoring the situation after the fixes have been done. Make sure your server logs the real IP in access logs and retains the logs for at least several months.<p> Log all HTTP traffic, if possible, to detect any remaining backdoors. Logging all traffic allows you to closely analyse what the attacker is doing and figure out the remaining fixes. On a Linux server you can log all traffic with a command such as sudo tcpdump -C 500 -z bzip2 -i lo -s 65535 -w http.pcap 'tcp port 8080'. It’s best if you’re able to log the traffic after HTTPS offloading (if you’re using Varnish or a similar HTTP cache); otherwise you need to do additional steps to be able to decrypt HTTPS traffic.<p> Check the logged HTTP traffic regularly with Wireshark for some time after fixing the hacked site to check for anything out of the ordinary. Analyse all exploit attempts, both successful and unsuccessful, to make sure no backdoors are left. Analysing the calls is difficult and tiresome, but pays off in the long run when you find the last backdoor that was left and avoid another fixing process or website defacement.<p> After you think all the backdoors have been fixed and attacks seem to be failing, make sure to check in periodically – the attacker might have left a backdoor that they didn’t use right away to be utilised in a few weeks, once the admin is no longer watching closely. https://www.zeip.fi/en/blogi/2013-12-13_first-impressions-jolla/ https://www.zeip.fi/en/blogi/2013-12-13_first-impressions-jolla/ Fri, 13 Dec 2013 23:32:20 +0200 I finally got my hands around the new, shiny Jolla – I might've waited for it a bit, since it took about twenty minutes from the SMS telling me I can pick it up to the time I was holding it in my hands... Now that I've toyed with it for a few days, I thought I might open up a bit on how this fresh saviour of the Finnish mobile phone industry feels like. User interface I instantly thought the user interface is quite stylish and good-looking. During the first start-up the user is presented with a short tutorial on the phone's controls, which basically means different swipes and the clues on the screen that tell you what kind of actions are possible in this view. I don't recall having this kind of tutorial in my earlier phone, Nokia N9, and I think it's a very welcome addition. There are clearly more ways to swipe than in N9; the functions differ based on if you swipe inside the screen or from outside the edge to the screen. This means that usage requires a bit of getting used to in order to get out of the old ways. Still, using the phone has become quite natural and I'm more comfortable with the control methods than with the N9. Software and functionality By default, only the store containing the Sailfish native software is shown. Unfortunately, this only contains a few software – although when I browsed the store just now I spotted many new applications that weren't there a few days ago, so it seems that the selection's constantly getting better. Quite a few of the near-standard applications of the phone are also separately installed via the store: Separate installation is needed for things such as calendar, calculator and the clock application. To clarify, these are indeed applications made by Jolla themselves, but it seems that they want to give the user the opportunity to not install any unnecessary applications – which for me sounds like a good idea! At this stage there are a few things lacking from Jolla's own applications also: For example the calendar application doesn't feature any kind of sync to other systems, which makes it pretty pointless, at least for me. I certainly hope that these kinds of shortcomings are fixed soon – there have been a few updates to the software already, so it seems that there is definitely hope! One of the good things about Jolla is also one often-quoted feature, running Android applications on the platform side-by-side with native Sailfish applications. This is a very necessary addition to the Sailfish native applications especially at these early stages of adoption. Installing Android applications happens by first installing the Yandex store to the phone and then using it to search and install any applications. The Yandex store has quite a bit wider selection of software, but unfortunately even it doesn't have anything. Officially Google Play store, which would be the place to go crazy with software, is not officially available for Jolla (although less surprisingly someone has already fixed this). Most of the applications found on Yandex work great: I already found Spotify, password manager aWallet and ebook reader Aldiko. Unfortunately some Android applications don't at least currently work as expected: Not one of the about ten call recorder applications I tried were functional, neither were a barcode scanner, voice recorder or the official Foursquare client. Conclusion In whole, I'm satisfied. The user interface is stylish and works for me, and even the functionalities keep on getting better. However I wouldn't recommend Jolla for the impatient users; I tried to move my contacts from the N9 to Jolla today, and first it looked that the sync worked just great – although I did spot at least one name I was unable to find even after something like five syncs.  There are some small shortcomings, but unlike with the N9 with Jolla one can find ease on the fact that the manufacturer has promised to actively develop the platform. I'm eagerly waiting for the improvements to come :) https://www.zeip.fi/en/blogi/2013-03-02_replacing-bind-powerdns-and-poweradmin/ https://www.zeip.fi/en/blogi/2013-03-02_replacing-bind-powerdns-and-poweradmin/ Sat, 02 Mar 2013 20:28:55 +0200 Why the switch? Recently I started wondering if there would be something better than the old, trustworthy BIND for all my DNS daemon needs. BIND is good, don't get me wrong, but I have to say that I don't feel it makes my job as an administrator for hundreds of domains any easier being (primarily) based on weirdly syntaxed plain-text files that are quite error-prone. So, encouraged by my not-so-recent discovery of Nginx, a very good alternative for the industrial standard Apache, I went on with my crusade to find a perfect DNS daemon. I found a variety of candidates that all seemed good and useable, but based on some further enquiries it seemed that BIND was one of the quite few that had enough users to make it stable. Enter PowerDNS, which has quite many users and seems to be commonly thought of as a more secure and better alternative for BIND. So I decided to give it a shot, especially after I found out that it has a bearable admin tool, PowerAdmin. One important point in deciding if I'd go on with the switch was finding out if it'd actually make my life easier. Security is a good point itself, but changing is also a lot of work. Here are a few points I based my decision on: Administration interface features templates, the changes of which can be applied to all the zones the template is used on. This was an important requirement; previously I was using ISPconfig as an admin interface, and although it, too, has templates, they are only applied when the zone is installed and any changes after that are not reflected on the zones. So any time I want to make a simple change to a specific set of domains, I had to do it one-by-one. Not good. Administration interface allows control of some zones be delegated to other users; so at least technically I can give my customers an user account with which they can then modify the zone themselves. Switchover was supposed to be easy, thanks to the zone2sql program that the kind folk of PowerDNS have made for us. And of course the security aspect was also important :) </ul> What's there to it? It's quite simple, really. But there were a few problems that I wasn't expecting, and that took a bit of time to resolve. It's supposed to be easier in PowerDNS 3.0, but since Debian Squeeze currently only has 2.9.something, I had to fight my way though that. Here's how. Start by installing PowerDNS. This was the easy part, so I'm not going to replicate that here. At least Debian has separate packages for the core and the storage module (which, for me, was MySQL); remember to install both! Then comes the (more) tricky part, migrating the data: First, find your BIND configuration file and use zone2sql to export the records into a SQL file: zone2sql --slave --named-conf=/etc/bind/named.conf.local --start-id=1 > records.sql Create domain insert clauses (this is a step that zone2sql apparently does itself in PowerDNS 3). Get the awk script at Zone2SQL FAQ.</dt> ./import2zone.awk records.sql > domains.sql Run the SQL scripts, the order shouldn't matter. </dl> After this you should have your data migrated and server set up. However, in my environment a few of the domains did not work right after the migration. This was because the name value of those domains (both in domains and records tables) included a dot after the domain name. I removed these by running one UPDATE clause for the domains table and one for the records table and tadaa, everything worked fine. All in all, it wasn't exactly difficult, but since some of these tips were a bit hard to find, I decided to write them up for future reference. Hope it's useful for someone else also! https://www.zeip.fi/en/blogi/2012-02-06_selecting-good-sms-gateway-finland/ https://www.zeip.fi/en/blogi/2012-02-06_selecting-good-sms-gateway-finland/ Mon, 06 Feb 2012 23:42:36 +0200 Today I realized I need to add SMS alerts to my dear Zabbix that tries to keep track of my servers' status. I just don't notice email alerts soon enough. However, when you search for SMS gateway from a search engine, there's something like a million results. Which one do I want to use? I tried a few of them, and tried to find one that was budget-friendly, flexible, fast and reliable. I know, I shouldn't expect to find one that actually fits all those – however I do think I found one that certainly seems to fit most of them. Here's my experiences on the ones I remembered. BudgetSMS With BudgetSMS I got off with a rocky start – I tried to use their API, but didn't get an SMS despite the OK reply. I actually gave up on it, but a few hours later I was looking through my Junk folders and found an email from BudgetSMS's staff asking for some info on my plans. It seems that BudgetSMS's services hadn't perhaps been used a lot to send to Finland; the support guy sent me some test SMS's through different routes to find out the best one. I got the service working fine, and they were also open to taking a smaller top up payment than officially offered on their web site. I loved the personal service and decided to stay with them. Clickatell Clickatell was the first one I found and tried. It seems to be quite big and professional, and the API choices are overwhelming. The worst thing about Clickatell is that you can't actually try sending real SMS's without investing on credit for the minimum amount of messages (which was 400 for me.) The test credit only sends a boilerplate test message instead of the message the user specified. Nexmo Nexmo was my second choice; I'm using it for my personal use since BudgetSMS is a bit more expensive for non-company use because of the VAT. The registration process and user interface are good and the prices are quite cheap. Nexmo also had the smallest minimum top up payment of the services I tried; though BudgetSMS was able to do the same when I asked for it. The others I tried a few others, and surfed on at least a dozen more web pages looking for a feasible one. Quite a few didn't work at all to Finland despite the promises, and some were ridiculously expensive for my use. I'm satisfied with my choices and believe both of the services I found are at least better than most of them. P.S. I also made a better version of the script, that can be used to send SMSs' from Zabbix. Currently it can only be found from the Zabbix JIRA, but I'm hoping that they also replace the old version on the Zabbix documentation with that one. https://www.zeip.fi/en/blogi/2011-12-17_last-dojo-more-testing/ https://www.zeip.fi/en/blogi/2011-12-17_last-dojo-more-testing/ Sat, 17 Dec 2011 21:06:09 +0200 The fourth, and last, dojo of our course was about Session-Based Testing. That was also (weirdly enough?) the title of the first presentation, that gave us a basic overview of session-based testing, the tester's roles etc. Once again I was late (I still wonder why all the site downtimes happen when you least hope for them...), but I got a good hang of it even seeing only a fraction of the presentation. The other presentation was a tool review of a few tools that could be used to facilitate session testing: RapidReporter and SessionTester (the latter being shockingly unpredictable with it's choice of name ;) The tools were usable, but I have to say that neither really caught my eye; both seemed quite lacking, with one of them (RapidReporter, I think) being just a text editor with some kind of parser for some of the syntax. I wonder if there are better programs out there for reporting session-based testing... After the presentations we divided into two groups: One tested our university's Web site again, while the other tested an Android app for browsing video services. I started with the Web testing. We organized the testing better this time: Each pair was assigned one site that they were supposed to test. We also discussed about good missions beforehand. We, with my pair, didn't use either of the reporting applications, mostly because they were in our minds quite lacking. Instead we settled for a traditional pen and paper tactic, which worked fine. We had chosen our Department of Mathematics Web site, and didn't really find anything interesting there. It was perhaps a bit too small a site to test for that long. After that we got to the perhaps more interesting case – an Android app called Fuugo. I have to say: Kudos to the guys for managing to code a very good app. I did found something strange once with full screen video + screen locking or something, but at least we didn't find anything repeatable that we could've reported as a bug. And we certainly did try... Though I have to admit that an Android app is quite a bit more difficult to test than an Web site, because my expertise is clearly more on the Web side of things (and of course the fact that browsers are one hell of a bughole...) It was the last dojo, and I have to admit I had fun. I learned quite a lot during the course – and I enjoyed the course structure much more than our regular lecture, demonstrations and an exam -type of thing. Nice work, guys! Next week we had a few guest lectures, too – a bit more about those later on. https://www.zeip.fi/en/blogi/2011-12-02_third-dojo-show-time/ https://www.zeip.fi/en/blogi/2011-12-02_third-dojo-show-time/ Fri, 02 Dec 2011 11:04:42 +0200 The third dojo's presentations were about Exploratory testing a web application and Bug management with RedMine. It was, finally, showtime – my group had done a presentation about the first, ET. It was quite a vast subject and we originally had some problems narrowing it down – especially since there isn't really too much sources about ET on a Web application out there. I don't know why, though... It seems to me like ET is THE way to do Web testing. Anyway, we managed to scrape together a usable presentation generally about ET and also on applying it to Web testing. I have to admit my presentation skills weren't really shining that day – For one reason or another I decided I just had to read aloud a slide that presented the differences between ET and scripted testing. God (only) knows why... Otherwise I think our presentation was good, though, and I did survive it despite my lousy performance. The second presentation is about a developer tool called RedMine – one that I've long thought I should try. As I thought, the tool seemed quite good and I will indeed need to take a closer look at it some time. After the presentation we set off to our actual testing dojo mission – the first two dojos were more of a coding type of thing, but this (and the next one) was just pure testing. Our mission was basically to test our university's Web site (and it's subdomains). I think the lecturers thought a little higher of the site... They promised a prize for finding a bug on the sites. Little did they know, the site's not that perfect... I have to admit, it was much less bugfree than I imagined, but there were a few weird things that I'd definitely call bugs, and as such both groups found more than enough to qualify for the prize – IMHO. In the end both groups were also tasked with reporting at least one of the found bugs to the webmaster. I personally think they should have all been reported, but I'm not sure what was done with them since I wasn't the one assigned to the task. Hopefully at least the most annoying ones were reported... https://www.zeip.fi/en/blogi/2011-11-29_second-dojo-ui-test-automation-and-watir/ https://www.zeip.fi/en/blogi/2011-11-29_second-dojo-ui-test-automation-and-watir/ Wed, 30 Nov 2011 01:24:58 +0200 The second dojo was about UI test automation and Watir. We started with the presentations – this time one presentation for the whole course. First was the general presentation about UI test automation. I have to admit that having already researched the subject to some extent because of my work, there wasn't that much new information in that presentation. However, the presentation was good and the lecturer's comments about UI test automation software, their problems and experiences about the software was nice. I also now remember which product I shouldn't buy to use for UI test automation... ;-) The toolkit presentation was about Watir, a web testing framework. I have personally researched Selenium, and it was very interesting to try and find the differences between Watir and Selenium, and trying to decide which one of them I should perhaps take into use also in the so-called ”real life”. I have to admit that Watir made a good impression, and I certainly decided to treat it as an equal with Selenium when trying to decide. Haven't yet made up my mind, though... After the Watir presentation, we moved on to the dojo section of the dojo, and split to the same groups again. This time we were supposed to take a Cucumber feature definition and make step definitions for it. One of our lecturers had kindly made an Web app that tried to fulfil the requirements that we had for the first dojo's application for which we wrote the feature definitions. This time our way of working at the dojo was more relaxed; also others besides the driver and copilot commented on the step definitions. This was a mixed blessing – on the other hand there was much more communication going on, which probably enhanced our performance as well as our ability to understand what was going on on the screen. On the down side, a few students were quite dominating in the dojo, and at some points the driver didn't necessarily have the slightest idea of the big picture on what s/he was asked to do. We had some serious problems trying to figure out the correct syntax for our step definitions – the tests seemed to pass even when they most certainly shouldn't have. In the end this proved to be an error in our step definitions indeed. This was a thing that I think everyone learned, since we tried fixing it for so long. In the end we managed to make a few working test cases with both definitions and step definitions. I'd say I learned quite a lot about Watir and integrating any testing tools to Cucumber – I think this will be of great benefit for me later on. https://www.zeip.fi/en/blogi/2011-11-29_first-dojo-coding-cucumber/ https://www.zeip.fi/en/blogi/2011-11-29_first-dojo-coding-cucumber/ Wed, 30 Nov 2011 00:53:56 +0200 I started my first dojo with being a bit late. I didn't have the slightest clue on where I should go or what I should do – always a nice beginning. At the first coding dojo we split to the two groups right from the beginning, one student from the expert group on the theoretical subject being in each dojo group. As we started with the presentations, I (along with others) found out that it was quite difficult to follow a presentation when there was another one going on on the opposite side of the (smallish) room – especially when the subject was the same. Despite that, though, the introduction to ATDD was interesting and enlightening. I knew something about the subject also beforehand, so it's hard to estimate how much the introduction was of use for me – at least the pros and cons became much clearer than before. The toolkit subject of the first coding dojo was Cucumber and Gherkin. This one was basically an on-hand introduction to the subject – no slides, just demonstration and good explanations about the subject. I had looked at Cucumber before and dreamed about some day maybe doing something real with it (especially after I realized there is a Drupal module that does something with Cucumber – cleverly named Drucumber). I liked the elegance and non-IT-people compatibility of the description files. It almost looks like you can just throw that at your client and s/he can instantly understand it. Because of the elegance, I had been wondering how exactly Cucumber works to achieve that. During the demonstration it became clear enough – it led me to believe that Cucumber perhaps might be more than just a cool idea and a nice toy for demonstration cases isolated from the Real World™. After the introduction, we finally got to work. One student (”driver”) on the laptop, one next to him/her (”copilot”). Originally the instructions were that these two students were also the only ones allowed to initiate discussions about the code – the only two exceptions were (at least if I understood correctly) asking about an action / line / whatever that someone didn't understand, or when either the driver or the copilot asked for help. This was a nice and on the other hand interesting idea, that proved to be somewhat challenging – some drivers were a bit shy and didn't tell the others what they were doing; on the other hand help was rarely asked and because of that some problems took longer to tackle than they maybe should've. I have to say that when I first heard about the structure and especially about the way only one person in each group would be on the laptop at one time, and everyone else in the group would be watching that, I was a bit nervous. It sounded like a sure recipe for a disaster including lockups and ”oh my god, I dunno, how embarrassing, I want to sink below the table”... Well, the first coding dojo luckily quickly alluded that fear to some extent. When we started working, I realized that the amount of new things concerning Cucumber, its syntax and its logic was quite manageable. Actually, when I sat on the driver seat it was quite clear for me what I should be doing. All in all, you could actually see the development in the students' understanding on Cucumber – we refactored the test cases multiple times, and each probably better than the last. In the end we actually had test cases that I thought were quite good, especially taking into account that we weren't really any kind of experts on Cucumber at the beginning. The dojo ended with some evaluation, and the problems (ie. having two presentations at the same time in the same room) were identified. My feeling after the first dojo was, that this was quite probably the best course I've ever been to. Nice! https://www.zeip.fi/en/blogi/2011-11-29_software-testing-special-course-101/ https://www.zeip.fi/en/blogi/2011-11-29_software-testing-special-course-101/ Wed, 30 Nov 2011 00:42:29 +0200 In this period I'm attending a course that I was very much looking forward to: Software testing – special course. The special part means that it's being lectured by two industry experts on software testing, and the structure of the course is quite different from any other course I've been to. I love the idea of industry people coming to universities as lecturers, because although the university staff are most certainly knowledgeable people on their respective fields, most of the courses have bee held by the same lecturer at least once before. That's all fine and dandy, of course, and it ensures the stable quality of the courses. However, it unfortunately also might lead to the courses being held in the same manner as they always have been, instead of trying something bold and interesting. What was so special about the structure of the course in comparison to the others? Well, the most important one: There were two lectures, on the first week. The lectures were mostly introduction to the course and to software testing basics. After that, it was all up to us. The main part of the course is held in four dojos, of which each holds two presentations from two student groups – one about a more theoretical subject and one toolset subject. After the presentations, the course splits to two groups and starts doing the actual stuff – using the tools that were just introduced, in a dojo-like manner. After the first lectures I was eager (and a bit nervous) to see what the dojos would really be like. Would I need to sink below the table and never come out? https://www.zeip.fi/en/blogi/2011-10-06_drupal-7-logging-rsyslog/ https://www.zeip.fi/en/blogi/2011-10-06_drupal-7-logging-rsyslog/ Thu, 06 Oct 2011 18:15:34 +0300 The usual way to do Drupal 7 logging is to save all the log entries to the database. This has it’s benefits and it’s problems: On one hand it works fine on every Drupal site without requiring any additional software or access to the server. On the other hand, it causes a few extra database queries (which in itself isn’t that horrible, but when there are already plenty, you usually want to do everything in your power to minimize the amount…). And of course, for the logs to be useful they should be monitored and acted upon. This might prove to be a challenge if you’re trying to maintain a multisite installation with tens of sites – you probably don’t have time to check all the sites’ logs every day via the web interface. The last point can of course be remedied by creating a custom tool that queries all the sites’ databases and gathers the data to one interface. I haven’t seen this kind of thing done ready, so it might be that you need to do it yourself. And usually you also have to add all new sites to one more system – which you’ll probably forget anyway. Ok, ok, what's your point? You can actually do this better – logging all the site activity without the small overhead and monitoring difficulties. This is possible via syslog (of course this usually requires that you host your own server – otherwise you probably won’t be able to set this up.) Syslogd is the server software responsible for all the logging your Linux system does. It logs something probably every minute: Incoming emails, outgoing emails, shell connection attempts, cron runs… You name it, syslog has it. It’s a highly dedicated tool that does one thing, and one thing only: Logs whatever something in your system wants it to log. Therefore it’s sensible to use syslog to log also your Drupal installations’ problems and errors. Less surprisingly, Drupal already has a module for this (too): The Drupal core includes a module called Syslog. So how do I do it? It’s quite easy to use: You actually just need to enable the module and go to admin/config/development/logging. Fill the following fields and click save, and you’re halfway done: Syslog identity This should be a string that you use to separate the installation from others that you have on your servers. I usually use the site domain name without the full stop, so www.zeip.eu becomes zeipeu. Syslog facility</dl> This is a kind of ”channel” that the syslogd uses to divide different software to different log files according to it's rules. Drupal only lets you use the LOCAL facilities – I myself use LOG_LOCAL1. LOG_LOCAL7 is reserved in some systems, so I don't suggest using it. </dl> When you've done this, you're already logging your Drupal errors to syslogd. But it might still not be saving them: It hasn't been told what to do with the facility you chose. So next, let's setup our syslogd. If you're using Debian, you probably have rsyslog as your syslogd, so I'll demonstrate the configuration of that one. If you're using something else, you'll probably want to use a similar config, but that one you'll have to figure out yourself. On Debian, create a new file as /etc/rsyslog.d/drupal.conf with the following content: $template RFC3164fmt,"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%- - - %msg%\n" local1.* -/var/log/drupal.log;RFC3164fmt The first row configures a new template for logging format, and the second row defines that all messages on facility local1 are logged to /var/log/drupal.log using the format. If you chose something else than LOG_LOCAL1, change the second row accordingly. Now you can log in to your site and after that check that your login has been recorded to /var/log/drupal.log. Then you can disable the ”Database logging” module to save a few queries. Congrats, you're done! https://www.zeip.fi/en/blogi/2011-07-05_extending-drupal-6-fineid-smart-card-authentication/ https://www.zeip.fi/en/blogi/2011-07-05_extending-drupal-6-fineid-smart-card-authentication/ Tue, 05 Jul 2011 12:19:50 +0300 On Sunday I wrote about Drupal 6 authentication with a FINeID card. In it I also claimed I was planning on expanding the Certificate Login module to facilitate an option for OpenID style identification, in which the user name doesn’t need to be taken directly from the certificate but the user can instead be attached to any number of certificates. This is possible by using Drupal 6’s authmap-table, which also Drupal’s own OpenID module uses. I spent a few hours last night to investigate the possibilities and managed to write a working extension for the module, in which the greatest shortcoming is that the user can’t inspect certificates attached to him/herself and can’t remove them. If the patch is accepted to the module, however, I will add that feature also. Meanwhile, here’s a short tutorial for anyone interested in taking matters in their own hands and using the patch. Code changes To use the new feature, you first need to apply the patch to the code. I recommend taking your module code from Drupal.org VCS according to the instructions, because the patch has been taken against it: $ git clone http://git.drupal.org/project/certificatelogin.git $ cd certificatelogin After this, download the patch from the issue. Move the downloaded patch to the certificatelogin directory that Git just created for you. Change to the directory and run the following command to apply the patch: $ patch -p0 < certificatelogin-authmap.patch patching file certificatelogin.module If the output differs from the above (and especially if the output has ”FAILED” in it), the patching probably failed. In this case you shouldn’t continue until you’ve successfully completed the patching. One more thing: Move the module to your Drupal installation to a nice and warm directory (e.g. sites/default/modules/custom). Settings When you’ve moved the module under your Drupal installation and enabled it, bear with me a bit more: Visit the module settings site (admin/settings/certificatelogin). If you’ve previously used the module, you’ll notice a new choice has appeared: ”Use authmap instead of user name.” Click it on. After this (save your settings and) visit path /login as a logged-in user. Click on the login button. If everything is as it should be, you now have a message ”Successfully added” and your certificate’s information on your screen. Now certificate login works without the user name having to be the certificate ID string. Using the method above, you can attach multiple certificates to your user account. Unfortunately, for now you can only remove certificates directly from the database authmap table. That’s it, have fun! https://www.zeip.fi/en/blogi/2011-07-04_drupal-6-authentication-apache-2-finnish-population-register-centre-vrk-fineid-smart-cardd/ https://www.zeip.fi/en/blogi/2011-07-04_drupal-6-authentication-apache-2-finnish-population-register-centre-vrk-fineid-smart-cardd/ Mon, 04 Jul 2011 21:33:53 +0300 Months ago I acquired the Finnish FINeID smart card produced by the Police and Population Register Centre, even though plans about discontinuing it altogether have already been made. In addition to being a traditional identification card, it has a certificate that can be used for both authentication and signing documents – although quite few services actually support it (pretty much the only ones are public sector and Itella NetPosti – one bank did allow it for quite a while, but they discontinued it stating lack of use). However, I’ve gained some interest on using it for authentication on my own services – I use e.g. my own OpenID service in quite a few places, so authenticating to that with a smart card would be quite cool (and, to be honest, that’s probably the only benefit that I would gain from that…) My OpenID service is based on Drupal, and I assumed that using a user certificate to authenticate to Drupal would be a piece of cake – there’s even a module for that (too.) Newsflash: It isn’t. Apache is like a small crybaby, the PRC root certificates are in the wrong format and life just plain sucks… To be fair, the authentication was quite easy to install in the end. You just needed to realize, that either Firefox or the fine smart card software by Fujitsu like to bug, so if the thing just doesn’t seem to work even when it should, restarting either your browser or your computer might just be the relief you need… (Figuring this out took me most of the hours spent on this project… The smart card worked for every other service, but for some reason it didn’t for mine until Firefox restart.) Installing client certificate authentication to Apache Start by choosing the root certificate you want/need. The PRC has both a generic root certificate (”VRK Gov. Root CA”) and a root for citizen certificates (”VRK Gov. CA for Citizen Qualified Certificates”). The generic root makes it possible to use any PRC-issued client certificates with your service – that is, also organization and health care professionals’ smart cards’ certificates. The citizen certificate root, on the other hand, assumably contains only the FINeID certificates issued in the electric identity cards. I personally used the generic root, even though my sites rarely have any users in addition to myself – mostly just because I couldn’t think of a reason not to use the generic root, and I trust all the PRC client certificates anyway. When you’ve selected the root you want, you need to figure out where you want to obtain it. You can either download it from the PRC site or, and this is a bit more secure, you can extract it off a electrical identity card you have, if you just have a Linux computer that knows how to use the card (which isn’t all that easy by itself… ;) ) Extracting the root certificate off an issued certificate better ensures that you actually do use the one and only PRC root instead of someone else’s forged version – after all you’ve probably fetched your card directly from the police station, so it probably is real. If you download the root certificate from the Web page, you should check the validity of the certificate by checking it’s fingerprint from an independent source. If you want to obtain the certificate by extracting it off the card, use the following commands depending on your choice of root: a) Generic root $ pkcs15-tool -r $(pkcs15-tool -c|grep -A4 "Root"|grep ID|cut -d: -f2) > vrkrootc.pem b) Citizen certificate root $ pkcs15-tool -r $(pkcs15-tool -c|grep -A4 "Citizen Qualified"|grep ID|cut -d: -f2) > vrkcqc.pem If you want to obtain the certificate from the PRC site, just download it. The certificate, however, is in the wrong format – PRC disitributes their root certificates in DER form, and Apache wants the certificates in PEM format. So you just need to convert it to PEM: $ openssl x509 -in vrkrootc.crt -inform DER -outform PEM -out vrkrootc.pem (If you used the citizen certificate root, the file name is vrkcqc.) Place the certificates in a good spot on your server. /etc/ssl works just fine. After this you just need to add a few lines to Apache’s SSL vhost file (in Debian it’s in /etc/apache2/sites-enabled): SSLCACertificateFile /etc/ssl/vrkrootc.crt SSLVerifyClient optional SSLVerifyDepth 2 SSLVerifyDepth value needs to be at least 2, if you use the generic root – in this case there is the abovementioned citizen certificate root between the generic root and the user’s certificate, so the user certificate’s ”depth” in the certificate chain is 2. If you use the citizen certificate root, setting the depth to 1 should be enough since the user certificate is directly in the given root. These should have information about user’s possible PRC certificate reach Drupal in the environment variable $_SERVER. Since the certificate is defined as optional, not required, also those not having / giving their certificate can access your site just like in the old days. Configuring the Drupal module When the Apache end is in good shape, you just need to have Drupal actually understand something of the authentication information given to it by Apache. This requires the Certificate Login module. When you’ve installed the module and enabled it, visit the module settings (Admin => Site settings => Certificate Login Settings) and define something like the following settings: Enable/Disable: Enabled PHP code to retrieve user name: ”$_SERVER[‘SSL_CLIENT_S_DN_CN’]” (without the outer quotes) Account creation: Enabled These settings have your certificate user’s user names as full name + SATU (Sähköinen asiointitunnus = electrical citizen ID). My personal goal is to expand the Certificate Login module to also be able to save the SATU in a separate field in the user profile, so that the user’s user name doesn’t need to be directly derived of the certificate. Also a port to Drupal 7 would be nice… Let’s see what I get done in the coming weeks. Until that, that’s pretty much all you can do with current software. ‘njoy. Troubleshooting Isn’t it working? Set Apache’s logging to something a little more extensive in the SSL vhost file, value debug should be good. Below are some problems I ran into: [error] [client 192.0.2.0] Certificate Verification: Certificate Chain too long (chain has 2 certificates, but maximum allowed are only 1) Adjust the SSLVerifyDepth value so, that it is equal or greater than n in ”chain has n certificates”. [info] SSL Library Error: error::SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a certificate No CAs known to server for verification? The client didn't pass a certificate. After a fresh restart of Apache, check from the error_log that it found the CA information: [debug] ssl_engine_init.c(598): Configuring client authentication [debug] ssl_engine_init.c(1193): CA certificate: /C=FI/ST=Finland/O=Vaestorekisterikeskus CA/OU=Valtion kansalaisvarmenteet/CN=VRK Gov. CA for Citizen Qualified Certificates Assuming this is ok, I got rid of this problem just by restarting my browser. More to read These are some of the resources I used while troubleshooting this. Unfortunately they're only in Finnish... Linux.fi-wikin sivu HST:stä Linux.fi-wikin sivu HST-kortin asentamisesta Apacheen Ismo Aulaskarin jo muutaman vuoden vanha ohje kortin käyttämisestä Linux-ympäristössä