liv: A woman with a long plait drinks a cup of tea (teapot)
[personal profile] liv
So my brother is trying to set up his own mail server as an act of resistance against the NSA. I have some doubts as to whether this is actually a worthwhile trade-off of political effect compared to effort, but anyway. He says:
I don't think I need massive expertise. I have a plan to set up a server and host email for myself and a few others and store stuff. I figured if I could teach my computer to send and receive email through mutt or sendmail, I'd have the skill. but before I start destroying people's lives by fucking up their emails, I thought I ought to learn how to do it.

Does anyone have any advice or can point Screwy to some relevant resources? I believe his computer dual-boots in Debian and Windows, but I wouldn't swear to that.

This is only related in the sense that I'm asking for advice from people with a particular area of expertise, on behalf of someone else. I have a really awesome new neighbour, who invited me in for a cup of tea and we ended up chatting for hours. Turns out her previous post was in Sweden so we had plenty to talk about comparing experiences. (Living here is a bit like being on a college staircase, in that all the other residents are also members of staff at the same university, so there's a strong assumption we will have a lot in common. I never really got to know my neighbours in my old place.)

Anyway, this neighbour is a bit disillusioned with academia, and is thinking of moving to the Civil Service. I know a few of you have made that leap, [ profile] shreena and [personal profile] lavendersparkle come to mind, and there's probably a couple more I've forgotten. Can you offer any advice for my neighbour? Please email or PM me if you don't want to comment on a public post. She's particularly interested in whether the Fast Stream route makes sense or even applies if you're 30 rather than straight out of uni. She's a geologist and would be interested in energy policy related stuff, and also wants the chance to travel to or even work in Europe.

(no subject)

Date: 2014-03-17 08:53 pm (UTC)
vatine: Generated with some CL code and a hand-designed blackletter font (Default)
From: [personal profile] vatine
So... At the minimum end, there isn't much to running a mail server. You ask your Debian system to install Exim, then you run the helpful configuration wizard. Then you generate an SSL certificate and enable opportunistic TLS, so that the mail going through Exim will be encrypted.

However, there are a few things that soon will crop up.
Spam is the first. Decent spam filtering is time-consuming, possibly expensive and demoralising.

Then there is keeping your DNS records in order.

And, yes, I have run both my own mail servers, as well as having run them as a day job.

It is a learning experience, that it's for sure...

Apologies if this egg-sucking lesson time :)

Date: 2014-03-18 12:14 am (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
In practice, I find doing greylisting (via Postgrey for Postfix in my case) removes the need to do any other spam filtering at the mail server level. Mail from new sources is told the server is busy and to try again later.

If you can never ever delay an incoming message from someone new for a few minutes, then greylisting is not for you. (But if it's that important, they should be using the phone or some other real-time system instead!) There is one bunch of losers I want email from who cannot manage to use an RFC-compliant mailer (they also bounce email to postmaster@theirdomain with 'no such user' which gives you an idea of their level), and there's an exception made for them. For everyone else, it really works.

So little spam gets through out of the thousands of emails every month reaching my mail client that it only has (looks) 29 things in the spam folder for the last four weeks: say one a day. My Gmail accounts' spam folders have more spam than that! At the same time, the server mail logs show endless delivery attempts from spambots that don't bother to redeliver delayed mail: dozens every minute is not uncommon.

Fortunately, not everyone does greylisting. If they did, the spambots would have to be rewritten to cope with it and it wouldn't work. But they don't, so they're not, so it does.

Re: Apologies if this egg-sucking lesson time :)

Date: 2014-03-18 06:02 am (UTC)
siderea: (Default)
From: [personal profile] siderea
Reeeeeeally. Fascinating. *takes notes*

Re: Apologies if this egg-sucking lesson time :)

Date: 2014-03-18 09:54 am (UTC)
vatine: Generated with some CL code and a hand-designed blackletter font (Default)
From: [personal profile] vatine
My experience with greylisting is that it removed (roughly) 20% of spam. Combining it with an MX of last resort pointing to an exising IP not speaking SMTP (my ADSL router, actually) cut another 20%. After that, I was still seeing 150-200 spam mails per day.

Re: Apologies if this egg-sucking lesson time :)

Date: 2014-03-18 07:42 pm (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
I am surprised. It's been the same level of 98+% effective for me since I started doing it, which was erm, back when I was using Debian Sarge.

Re: Apologies if this egg-sucking lesson time :)

Date: 2014-03-20 03:12 pm (UTC)
vatine: Generated with some CL code and a hand-designed blackletter font (Default)
From: [personal profile] vatine
I used to use the mail address(es) that ended up in my mailbox on Usenet and the oldest harked back to 1993. It probably was in milliplicate in all spambooks.

(no subject)

Date: 2014-03-17 09:11 pm (UTC)
jack: (Default)
From: [personal profile] jack
Yay, screwy! Using encrypted email, whether or not on a private mail server, might be the most useful thing to do for privacy. I don't actually know how to start, but I think you can do it with some hassle installing software but no system administration.

(no subject)

Date: 2014-03-17 09:13 pm (UTC)
lethargic_man: (computer geekery)
From: [personal profile] lethargic_man
As [personal profile] vatine said, don't use sendmail, use exim. sendmail's config file looks like R2D2's diary.

I've used my desktop as its own mailserver (it's easy in Linux), but in recent years many mail relays will reject mail from a dynamic IP address as likely to be spam. Unless Screwy can get a static IP address—and possibly that would not be enough either—he may run into this problem.

(no subject)

Date: 2014-03-17 11:48 pm (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
By coincidence, I have just been setting up a Debian server. I prefer Postfix to Exim: the latter had a truly horrible security issue in the recent past which caught some people I know, and I've been using Postfix for ages anyway.

I did need to remind myself about how to set it up - it was years since I had done it, and I'd done the change between Debian 6 and Debian 7 in the meantime, which is different to setting up a new Debian 7 one. Just copying the config files from the old server, and changing the hostname didn't work, so I found a series of recipes that did and got me Postfix handling the SMTP and Dovecot doing the TLS verification and POP3/IMAP side.

(no subject)

Date: 2014-03-17 11:54 pm (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
Oh, this is better done on a server somewhere - only being able to send/receive mail while your PC is switched on and running the right OS is a bit of a pain.

The good news is that can now mean as little as £36 a year for your own imaginary PC with more disk space and bandwidth than he will ever need for this. Depending on his security needs, the disk space can be encrypted and if it's just used for email, it can be firewalled against everything except that.

(no subject)

Date: 2014-03-18 06:06 am (UTC)
siderea: (Default)
From: [personal profile] siderea
The good news is that can now mean as little as £36 a year for your own imaginary PC with more disk space and bandwidth than he will ever need for this. Depending on his security needs, the disk space can be encrypted and if it's just used for email, it can be firewalled against everything except that.

I would like to know everything you'd care to share about:

1) Where you are getting a virtual server for £36/yr.
2) Encrypted disk for an email server; how's that work and what do you use for it, and what's it proof against?

(I've been thinking hard thinky thoughts about secure email lately, for professional reasons.)

(no subject)

Date: 2014-03-18 11:06 am (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
1. It's Digital Ocean* who charge $5 for 28 days** for a virtual server that will certainly act as an email server with no problems. At the moment, $5 is almost exactly £3*** and it works out as a quarter of what I was paying until I discovered just what economies of scale has done to VPS prices.

You can pay more for more RAM, but you don't need to. You could do more (host websites etc) on the smallest size, but the fewer things it does, the more secure it is going to be.

You have the choice of having it in New York, San Francisco or Amsterdam. I've gone for the latter, on the basis that it's closer to me and whoever is tapping the datalinks into the hosting facility may not include the NSA.

2. You'd use TrueCrypt. This will prevent anyone looking at the disk from finding out what's on it unless they can control the running server's CPU or RAM.****

In practice, I don't bother. Hardly anyone encrypts email, and if you use PGP-style encryption for email conversations, it's safe until it hits your computer. Even if they are controlling your server's CPU/RAM.*****

For non-encrypted email, if anyone with an acronym wanted to read it, they would tap / are already tapping the connection to the hosting facility, and that's going to give away your email usage patterns anyway...

Who do you want it to be secure from?

* Affiliate link: if you use them for two months, I get two months free.

** Actually 0.7c an hour, but you lose the IP address every time you take your VPS offline, so this pricing only makes sense for people testing stuff before putting it to work on a lasting one.

*** I was going to say that you'd need a credit card that doesn't penalise you for using dollars or a PayPal account, but I see you're in the US! :)

**** Which they might be on any server you don't physically control.

***** But if you are of that much interest to them, they could be sitting outside your place reading what's on your screen via the RF emissions of your graphics card / monitor and/or have hacked your computer and/or have bugs watching or listening to your keystrokes. Or, if they don't mind using torture, just doing that until you talk.

(no subject)

Date: 2014-03-23 12:53 am (UTC)
siderea: (Default)
From: [personal profile] siderea
(At the risk of hijacking Liv's journal -- Liv, do we need to move this elsewhere?)

Hi, I'm back. I was inadvertently delayed, sorry.

1) Oooh, thanks for the tip.

2) I'm a regular TrueCrypt user at the desktop. My primary threat model has not been the NSA -- I'm a healthcare professional with highly confidential and prejudicial medical files entrusted to me, so mostly I was worried about someone stealing the device they were on for the value of the hardware, and then discovering the data bonanza; I also have to be a little concerned about PIs (possibly highly-skilled amateurs) engaged in espionage in the service of legal proceedings or stalkings. Sometimes I handle information of supposed relevance to divorce and child custody proceedings, and sometimes I handle information of supposed relevance to divorce and child custody proceedings of computer scientists. I would like to, let us say, reduce temptation: if you want what's on my HD, you can damn well explain it to a judge and do the Dance of the Subpoenas.

Here, I am primarily concerned with providing increased security for my patients, and providing them with secure-er ways of emailing me.

So I started thinking, "wouldn't it be groovy to have my email files actually live[*] on a TrueCrypt volume"; I looked into it. And I stopped. Because... how would that work exactly? Mail comes in via SMTP, and, well, how does it get into the TrueCrypt volume? The TrueCrypt volume just lives mounted on the device? How is that any more secure than not bothering to use it? Doesn't that then reduce the security of the TrueCrypt volume to the security of the server? It is wide open to anyone who can get into the server, and doesn't keep anyone additional out, yes? Isn't it like having a splufty un-pickable locking file cabinet, that one props open all the time?

Or does the TrueCrypt volume somehow get opened just for an MTA to write into? If so, how is the password managed? If the password is known to the MTA, isn't that like putting the key on the top of the file cabinet?

I don't really get encryption, so I'll readily confess I may be missing something here.

Have you actually tried doing this?

[* Where "live" means "in either a nice /var/mail spool format, and/or in nmh format after running through my ~3000 lines of procmailrc.]

Meanwhile, the whole Virtual thing has me twitchy. I have no sense whatsoever of how secure a VPS is against (1) the nice sysadmins of the company that is selling me the VPS, or (2) the nice sysadmins of the company (e.g. Heroku) selling the cloud space to the nice sysadmins in (1).

I realize I may be stuck colocating if I get serious about this.
Edited (Now with even more question marks.) Date: 2014-03-23 12:55 am (UTC)

(no subject)

Date: 2014-03-23 10:51 am (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
Typically, with a server you get the equivalent of a terminal attached to a serial port. Exactly how it's done varies, but for this it means you can enter the required passwords on start up. This is a reason to have an alert system set up, something that goes 'Erm, I've tried talking to the server and it's not answering', because if the server falls over for any reason, you need to do something to restart it.

Yes, you need to trust the providers of any VPS. If they wanted to read your disk / RAM / etc, they could. Even with your own locked up hardware somewhere, something could be sniffing the network connection, able to read any unencrypted email. Because of these, I don't bother with TrueCrypt on my VPSes.

You are already doing more than most, and the most useful thing you can do is get your clients to use PGP to email you.

(no subject)

Date: 2014-03-23 10:54 pm (UTC)
siderea: (Default)
From: [personal profile] siderea
Typically, with a server you get the equivalent of a terminal attached to a serial port. Exactly how it's done varies, but for this it means you can enter the required passwords on start up. This is a reason to have an alert system set up, something that goes 'Erm, I've tried talking to the server and it's not answering', because if the server falls over for any reason, you need to do something to restart it.

Yeah, I'm familiar with the basics of having a server, both virtual and the one I have under my desk. But, in the scenario you describe:

1) Let's say you have 99% uptime. That means that if your email spool is on a TrueCrypt volume, ninety-nine percent of the time the volume is wide open, unencrypted. The only time a TrueCrypt volume is protected against unauthorized reading is when it's not in use. A server needs it in use all the time. I don't understand, given this, why one bothers using a TrueCrypt volume at all.

1a) Do you have any idea what can happen to a TrueCrypt volume when the host on which it's mounted crashes?

2) If your TrueCrypt-using SMTP/[IMAP|POP] server goes down, and the upstream SMTP server can't pass it its mail through TSL/SSL... what happens to that mail while it's waiting the statutorily-up-to-3-days for you to restart your server? It sits on an unencrypted drive?

So if somebody wants to read your mail spool, they just have to DoS your machine so your mail backs up on a more accessible server?

It sounds to me that even when you own your own hardware, there's absolutely no benefit to using TrueCrypt for a mail server, and quite a lot of downside, in terms of labor and increased risk in disaster recovery.

Now, it seems to me that if you're willing to be typing passwords into the server on every reboot, the best approach isn't TrueCrypt, it's a pub/priv keypair scheme, where your MTA takes every incoming email and lovingly wraps it in your (dedicated to this purpose) public key, and shoves it on a queue, and then it sits there waiting for you to run an intermediary between MTA and MUA which unencrypts the spool for the MUA.

Of course, the very best thing would be an MUA which doesn't need the messages unencrypted for it. Man, I'd love a version of nmh, all the components of which were bright enough to unencrypt message files on the fly. But then, there's the question of whether Every. Single. Command. would then require the user to type in a password, or whether the private key remains in some sense unlocked for a while.

the most useful thing you can do is get your clients to use PGP to email you.

1) PGP can't secure the metadata, which frankly is the most prejudicial information in the message.

2) Ha! That will be a cold day in hell. The vast majority of email users use systems which are fundamentally incompatible with key-management -- they're all web-based, either hosted services like Gmail or web-interfaced IMAP like Roundcube. While there are third-party services who will give you a browser plugin or greasemonkey script or some such to allow you to store your private key(s) on their server so you can read encrypted emails sent to you... holy crap, you're putting your private keys on some random anonymous person's server! :/

(no subject)

Date: 2014-03-24 01:14 am (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
AIUI, if system encryption is set up correctly, unless you are either logged in with sufficient rights or have very low-level access - like reading its RAM - to the running server, you can't read the contents of the encrypted data. To the point where, AIUI, people running servers containing very dodgy stuff use it and have disposable public-facing sites run encrypted links to it, knowing that the hosting place for the main server can't find out what's on it, even by sniffing the network connection.

But you can't guarantee no-one has that low-level access, especially on a VPS running a hypervisor.

I wonder if you either need to accept that you're doing the best that you can - no-one can read your disk without your co-operation, but they could sniff the network at any point between you and the client - or get clients to use something other than email to communicate with you.

(no subject)

Date: 2014-03-24 02:04 am (UTC)
siderea: (Default)
From: [personal profile] siderea
AIUI, if system encryption is set up correctly, unless you are either logged in with sufficient rights or have very low-level access - like reading its RAM - to the running server, you can't read the contents of the encrypted data.

What do you mean by "system encryption"? Do you mean storing data/mail queues on TrueCrypt? Or are you now talking about something else?

To the point where, AIUI, people running servers containing very dodgy stuff use it and have disposable public-facing sites run encrypted links to it, knowing that the hosting place for the main server can't find out what's on it, even by sniffing the network connection.

Ah. If you ever encounter one of those people, do please send them my way so I can ask them some technical questions.

(no subject)

Date: 2014-03-24 08:10 am (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
TrueCrypt's documentation page on 'system encryption'.

Contacting that sort of person is left as an exercise for the reader :)

(no subject)

Date: 2014-03-18 07:51 am (UTC)
lavendersparkle: Jewish rat (Default)
From: [personal profile] lavendersparkle
I think that your geologist friend has several options and she could pursue some or all of them simultaneously.

One option is to look on the Civil Service Jobs website, where all civil service jobs that are open to people who are not currently in the civil service are advertised.

The next option would be to try to enter through the faststream. There are certainly people who enter the faststream in their 30s after having done something else first; I know several, including someone who entered after a Material Science postdoc. There are several different types of faststream based on different specialism. You can apply for more than one at the same time and if you don't get in under one you might still get in under another. The starting salary for a fast streamer is usually a bit under £30000 a year. One thing she might want to investigate which may not be obvious would be to apply for the operational research faststream, which requires that at least 50 percent of your undergrad was in a numerate subject. There seem to be a lot of people with science PhDs amount the operational researchers I've worked with. She might also be eligible to apply as a Statistician depending on the make up of her undergraduate degree.

However she applies, it would be useful to take he Civil Service Competency Framework into account in her applications and interviews.

My final bit of advice is 'do it'. In the Civil Service you can work 9 to 5 and not be expected to dedicate your every waking hour to your work. You can work part time without destroying your career prospects. You can take a full year's maternity leave without having to worry about what it will do to your research rating. You have job security and a defined benefits pension. Your work will be read by the people who run the country, not just a few reviewers. The civil service is full of former academics who realised that modern academia just isn't worth the candle. This little rat is very happy that she left the ship.

(no subject)

Date: 2014-03-18 08:33 am (UTC)
From: [personal profile] quizcustodiet
This is a really good summary, so all I can really add is a couple of points of nuance.

One point to bear in mind is that all Fast Streams are not created equal, so one should consider one's ultimate goal in choosing what they apply for. In my experience, the newer 'Fast Streams' (specifically HR, IT in business and Operational Research) can sometimes give you quicker promotion, but they are quite narrow in focus and (in my experience) they're not viewed as being as prestigious as the generalist Fast Stream (including the Science and Engineering Fast Stream).

General promotion in the Civil Service is still about generalism rather than specialism, so the broad experience that the generalist fast stream gives you is the best preparation for future promotions. If your friend wants to be a Permanent Secretary, she'd probably be better off with the generalist stream.

Having looked for science jobs in the Civil Service, there are a surprising number which are agnostic on one's scientific discipline (back to the generalism again!). I haven't specifically seen geology jobs, but I'm sure there will be some in DEFRA and DECC. Your friend might also want to look into job in Government but outside the Civil Service, e.g. in the Environment Agency.

On entry route, the major advantage of the Fast Stream is that it is the widest gate: very few Civil Service jobs are advertised externally, so you can end up waiting a long time to try to get in directly. She wouldn't be that unusual, either: the average age for FS entry has been rising recently, and was around 27 when I joined.

Another option that's worth considering is direct entry at the Treasury. You enter at the same level as the Fast Stream, and get less focussed support, but the Treasury is quite a prestigious place to work within Whitehall, and it means that if you make a mistake at the Fast Stream assessment centre you've got a backup plan.

(no subject)

Date: 2014-03-18 06:44 pm (UTC)
lavendersparkle: Jewish rat (Default)
From: [personal profile] lavendersparkle
Gus O'Donnell is proof that you don't have to start as a generalist to get to the top. As is clear from the comments, the big issue is finding a way in and once you're in you have a lot more options for moving around. I joined the civil service as an economics faststreamer, so that and the other analytical faststreams are the ones I know most about. It is unusual to move into a generalist grade 7 post straight from an analytical faststream, but I know a lot of analytical fast streamers who have done generalist posts as part of their faststream. I also know a couple of former faststream economists who have moved into generalist posts after they got to grade 7 and are on the way up.

A difference to take into account is that the generalist faststream is now very structured and run by Cabinet Office whereas the analytical faststreams are more of the dump you in a department and expect it to put a bit more effort in your development approach. This makes the analytical faststreams more flexible but the generalist faststream gives you more experience in more different departments.

The Treasury has the worst pay and the longest hours of Whitehall. It does have a certain kudos. Treasury makes a habit of sending someone at least one grade, probably two below the people from other departments that their having a meeting with, so it gives you lots of exposure to senior people.

(no subject)

Date: 2014-03-18 10:53 pm (UTC)
From: [personal profile] quizcustodiet
Apologies, you're quite right - I meant to specifically include the economics Fast Stream in the same breath as the generalist, but it was very early this morning! It's definitely as respected, and I believe is somewhat better paid, but sadly is probably not an easy option for a geologist.

I agree on the pay and the hours at the Treasury, but it attracts overwhelmingly young, bright and motivated people, which I enjoyed on my secondment there. This may have been good luck, but it also had the best team dynamic and social vibe of anywhere I've worked previously.

I mainly wanted to offer it as another way in, as the Fast Stream is so competitive that it only takes one bad day to miss the sift, and as others have highlighted it's easier to get the job you really want from inside the Civil Service than outside.

(no subject)

Date: 2014-03-18 01:26 pm (UTC)
emperor: (Default)
From: [personal profile] emperor
My experience of civil servants is that "you can work 9-5" is far from universally true - quite a few seem to end up working more than that, and I get the impression that to become senior you have to be prepared to work distinctly more than that (presumably because MPs work funny hours).

(no subject)

Date: 2014-03-18 10:43 pm (UTC)
From: [personal profile] shreena
It's definitely the case that some CS posts require you to work long hours - e.g. the Bill team post I did recently requires long hours because Parliament sits late, often. The Lords technically can go as late as they want and sometimes do, even the Commons sits till 10:45pm or so which is brutal if you have also had to get in at 9am to prepare Ministers' speaking notes.

However, I think lavendersparkles' point is valid in that there are plenty of jobs where you can work 9-5 and still get taken seriously. I really like that I have encountered various senior people who are rigorous about leaving on time to collect their children from nursery/school.

That said, as in any serious professional career, there will be the odd crisis where - if you don't come in and deal with it - there will be consequences for your career. But, if you pick the right post, that's very rare. [I kind of want to give some examples here but they would be indiscreet.]

(no subject)

Date: 2014-03-18 08:33 am (UTC)
ewx: (Default)
From: [personal profile] ewx
I use Exim, Spamassassin and Dovecot for mail transport, spam filtering and IMAP service, respectively. All are in Debian. There are other options.

AFAICT opportunistic TLS on SMTP connections can only provide any confidentiality against completely passive attacks; and even that's only true if all parties involved support it, which isn't in general true; and there is a lot of active interception going on.

OTOH if the aim of the exercise is a political statement rather than a technical outcome then I suppose its effectiveness depends on the audience the statement is being made to!


Date: 2014-03-24 09:05 pm (UTC)
From: (Anonymous)
Hey all,

I've guess I've come to late to get specific help. It serves me right for not reading my sister's blog enough. Here are a few things:
1. Why I want to run my own email.
This is ill thought out, but basically something has to happen in response to the cast iron proof that mass surveillance of electronic communication has become routine. It occurs to me, in my ignorance, that the tools exist to beat the crude techniques of the NSA, GCHQ and others, but most involve a sea change in the way we use the net (in particular using encryption and remailers). In addition the net has been taken over by giant companies whose business model is: offer stuff for user data. I kind of always felt uneasy about that tradeoff, but took it any way because I know nothing about computers. I hacked around in BASIC 20 years ago. That's it. I didn't even realise I could read gmail in thunderbird. I figured it was a website. It seems clear to me that google, microsoft, yahoo etc collecting data is, at the very least, making state surveillance very much easier. So, I have two motives: move away from commercial services, and to learn how this shit works. I figure my ignorance makes me a sitting duck. Nothing I can do can prevent the state gathering data on my habits, because I write to people who use gmail, etc. However, it is a network, and I'd like my bit in it to be accessible only to me. That has to be possible because Snowden said so (bad argument, but if he's going to risk his life, I can try to take control of my data).

2. What I want. I want some server space, preferably my own rather than a virtual one, which will send, receive and store email, host some documents and act as a VPN. I want my own server because at the mo this lot,, provide my email. I'm using POP (is that the right lingo?) to get the email to my computer. So, I figure that the setup is isomorphic to using a VPS. Presumably document storage is easy, and I can use filezilla or equivalent to get the files there and back securely. I haven't thought beyond using openvpn to have my own VPN. The last bit is quite important to me because my ISP is commercial and is logging my internet activity. Having said all of that, I've genuinely no idea whether any of that will help me achieve my non-learning based objective.

3. I'm rubbish at this. I can't even configure mutt and exim4 to send and receive email via a smarthost (I think that's the right term). I tried. Emails I send disappear and never seem to arrive. Still it is an improvement on their not coming back at all. I sort of get the big picture, a program like exim 'listens' for incoming mail, sticks it in a mail box (or is that done by a different program?), a 2nd/3rd program will move the mail to my computer (or indeed any properly configured computer), finally, if I get it right, I can authorise the first program to send email from me to any old domain but not to fire spam off all over the net (I do have a good line on cheap viagra though, if you want to keep her satisfied for longer). I have no idea what a DNS record is.


PS. Why is spam such a problem? Surely (the surely operator is a dangerous operator), exim/postfix what have you is configured to require a password to ping emails off over the net? Is it that I'm going to end up receiving the spam?

Re: hmmm

Date: 2014-03-25 09:16 am (UTC)
ewx: (Default)
From: [personal profile] ewx
IMHO you need to sort out point 3 before even starting on 1 and 2. Hopefully someone can recommend a good text on Internet architectural basics; someone can tell you things like "DNS maps names to addresses" but there's more to it than that and the details matter when you're running mail systems or worrying about threats to confidentiality. (I'm honestly not sure what the best starting point to recommend is.)

To address some specific points:

If outbound email is "disappearing", you need to look at the logs on the machine it's last known to have reached to find out what happened to it.

Spam is largely sent from compromised systems and accounts. The chances are that, yes, you are going to receive spam. If you don't see it at the moment then either you are quite lucky or you are getting the benefit of your provider's spam filtering.

Re: hmmm

Date: 2014-03-25 09:44 pm (UTC)
From: (Anonymous)
I think your HO is correct (sorting out 3 will help sort out 1 though). I'll check out logs and swot up on the basics some more. Thanks for the help.

Re: hmmm

Date: 2014-03-26 05:02 am (UTC)
siderea: (Default)
From: [personal profile] siderea
Take this, you're going to need it where you're going.


I'm using POP (is that the right lingo?) to get the email to my computer.

Might be. The two choices are basically POP and IMAP.

I sort of get the big picture, a program like exim 'listens' for incoming mail, sticks it in a mail box (or is that done by a different program?)

exim is an example of a Mail Transfer Agent (MTA). MTAs are programs which transmit emails between computers systems on the Internet. As you say, your MTA will listen for incoming email for you -- more accurately, for your domain(s) and receive it. It will either put it in a mail box, or, if you prefer, you can plug it into a Mail Delivery Agent (MDA), such as procmail. A MDA allows you to do more sophisticated and interesting things to your incoming email. (I am a procmail geek. I have, last count, something like 2.5k lines of custom procmailrc files, and I've been known to joke that my procmail config almost smart enought to answer my emails for me.) The MDA puts emails in boxes.

Then the Mail User Agent, which is something like mutt or Thunderbird, reads the email in the boxes.

An analogy may help. I rent a mailbox from a private mailbox company. When someone sends me mail there, it goes through the regular US Post Office, to the local Post Office (my MTA), which delivers it to me at my address -- that is, the address of the private mailbox company (my MDA). The private mailbox company accepts my mail, and puts it in my mailbox. I can then come by any time to pick up and read my mail (I am my MUA).

I could have, instead, had my mail box at my house (have the MTA deliver right to where my MUA can get it on my local machine), or I could have rented a mailbox at and from the Post Office, and not involved a third party at all. But this way, I can call up and ask, "Do I have mail? Who is it from?" and the nice guys there tell me.

Surely (the surely operator is a dangerous operator), exim/postfix what have you is configured to require a password to ping emails off over the net?

An MTA (that is, an SMTP server) which does not require a password is called an "open relay". Open relays are frowned upon, because spammers.

But if you are a dastardly spammer, and you're already cool with breaking laws, you could just break into someone else's computer and set up an open relay on it, and use your victim's computer to distribute your spam for you.

Or if your victim's computer is configured to be allowed to send out-going emails through an SMTP server it's authenticated against, use that.

Re: hmmm

Date: 2014-03-26 10:34 pm (UTC)
From: (Anonymous)
Well thank you very much. The first link looks like just the thing I've been looking for and the second saves me doing the obvious. Your explanation is also very helpful. Woot, fucking up the internet, here I come (perhaps a little arrogant, but I'll probably add to the mountain of spam).

(no subject)

Date: 2014-03-18 08:41 am (UTC)
From: [personal profile] shreena
I love the civil service and, every now and then, I think to myself "this was SO the right decision".

I love it because of the variety, the interestingness of the work, the people (not all the people, obviously but many of the people), and the atmosphere. To expand a bit on the atmosphere, I absolutely love that (unlike what a lot of my friends in the private sector tell me) it is perfectly possible to rise to the top of the civil service as a woman without ever having worn make-up or heels, it's a place that is genuinely accepting of diversity and tends to look at your work rather than the way you present.

However, some of what I love about the civil service, she might not. I am a bit shallow and fickle, I like flitting about and doing completely different policy areas and I really like that I don't have to be an expert to advise a Minister. Lots of academics really like being an expert but being an expert is - to be blunt - not really that respected in the civil service. Experts in particular policy areas don't commonly rise that high. Conversely, generalist skills like being able to communicate clearly with a Minister and boil down a complicated topic to a few words are valued very highly. Some academics have that sort of skill set but some don't.

I think, also, something that some people find tough is that experience outside the civil service - unless it is very directly relevant or you are very senior - will not generally count for much within the civil service. Even in a Department like DECC, I would be surprised if having been a geologist at post-doc level would give you much of a headstart over someone who'd done geology at undergraduate level. What I'm trying to say is that she would be starting over almost completely and you can only do that successfully if you don't resent that.

In terms of practicalities, the civil service has been shrinking so there aren't very many jobs. Civil service jobs are usually advertised first internally and then externally which often means that the ones that end up externally advertised are not that great. However, it can still be worth taking something that you're not wild about on grounds that you then get access to the internal jobs market and can move.

I did do the fast stream and started when I was 30. There are plenty of fast streamers that age. The fast stream is a great way to get a sense, fairly quickly, of what the civil service is like, you get training and support and - crucially - everyone knows that you're new so won't assume you already know the ins and outs of being in government.

However, when I did it, it was a lot more flexible and you got to choose what postings you did and when you went for promotion. I, therefore, got promoted fairly quickly (just under 3 years) which worked for me - I already had plenty of experience in the workplace and I didn't want to hang around. Now, you get much less choice in what you do and it is a structured 4 year programme across the whole of Government. So, if she specifically wants to work on energy policy, the fast stream is quite a slow (no pun intended) way of achieving that goal as it'll take her 4 years to get there.

If she's looking at non-fast stream jobs on the CS website, she should look at HEO (salary c.30k), SEO (salary c40k) and Grade 7 (c50k) jobs. For info, HEO is about where you start as a fast streamer and 7 is the grade you're generally promoted into. It may take some time for jobs she's interested in to come up - given the general lack of jobs at the moment - and it may well take a few applications to get the hang of the competency based forms. Although the HEO/SEO salaries are relatively good, the SEO band in particular is (IMO) the most overpaid band in the civil service as you are still - if you're in a policy job - very heavily managed and have much less independence in your work than you would usually expect of a job at the 40k level. However, given that the civil service isn't great at valuing outside experience, she may well have to go for something at the HEO/SEO level to get her foot in the door.

Or, for a shorter summary of the practicalities, when you're coming in from outside, you often have to play the long game with the civil service and do jobs which aren't necessarily ideal for a bit to get into the more interesting ones.

(no subject)

Date: 2014-03-18 09:09 am (UTC)
synecdochic: torso of a man wearing jeans, hands bound with belt (Default)
From: [personal profile] synecdochic
I've been running my own email for fifteen years or so, and it doesn't really require much attention. Getting it set up is the biggest pain.

The question of what MTA is the best has been starting fistfights for years. They all suck. I used to use qmail, because it was written by a rabid weasel on crack but it was simple, easy to configure, and well-secured; these days I generally use postfix when I need to set up a new one.

The biggest issues he'll find is that Gmail, being the nine thousand pound gorilla of email, can randomly decree they're going to enforce something and then bam, all of a sudden you can't send email to gmail addresses until you figure out what the fuck they're on about today.

small how to

Date: 2014-03-18 10:04 am (UTC)
From: (Anonymous)
One of my friends actually did it some time ago, he laid down the broad concepts in a blog entry:



(no subject)

Date: 2014-03-19 01:52 pm (UTC)
thekumquat: (Default)
From: [personal profile] thekumquat
Hi, pointed here by . Firstly I agree with all Shreena has said.

I'm an ex-academic who joined the Fast Stream in my late 20s. I would recommend it as a route to the most interesting jobs in the CS, and while most people take 4-5 years to get promoted, the ones who do it more quickly are invariably older with outside experience, usually of skills like financial management and people managment.

The new FS scheme has placements of just 6 months in the first 2 years, which all of us who have been there longer find odd as usually we would be finally getting to grips with a complex policy area (there's no other kind, actually) in a 1-year placement. It's been running for a year now so will see how that pans out.

There are issues (PM me if you like) but it's certainly more diversity-accepting than most places, though as with anywhere, more depends on your individual manager and team than anything else.

Also, Ofgem seem to be constantly recruiting and like geologists, which is another way to become a quasi-civil-servant.

Thank you

Date: 2014-03-31 07:00 pm (UTC)
From: (Anonymous)
Thanks to all of you who replied to the request for information and advice on jobs in the civil service. I am the new neighbour mentioned in the above entry, and yes, I am looking to escape from academia...

All the information and advice you provided has been very helpful. It seems that the civil service is a great place to work and I got a lot of very positive vibes from you all.

In the end I still have time on my current contract, so thankfully there is no need to rush. Instead I have time to make an 'educated decision' as to what will be best for and all your points have really helped! Thank you! :-)


Miscellaneous. Eclectic. Random. Perhaps markedly literate, or at least suffering from the compulsion to read any text that presents itself, including cereal boxes.

Top topics

March 2017

56 7 891011
12 1314 15161718
1920 21 22232425

Expand Cut Tags

No cut tags

Subscription Filters