Nelson will hold the post alongside his existing position as the Ministry of Justice (MoJ) CIO and will take over from Joe Harley, who will be retiring from the civil service at the end of March.
Nelson has been heavily involved in the government's cloud computing plans over the last year.
Commenting on the appointment, Ian Watmore, permanent secretary at the Cabinet Office, said: "It is fantastic to be able to assign the role of government CIO to someone who has held major CIO roles in private sector and has been involved in the ICT strategy since the very beginning. Andy has worked closely with Joe over the past months and will continue to do so – ensuring that we continue to deliver ICT services fit for a modern civil service."