Analysis: What MPs say is wrong with spy bill
Everyone agrees a new law governing surveillance powers is required but the devil is in the detail, as a series of parliamentary reports have illustrated.
The existing law, all agree, is complex and lacking in transparency but all the signs are that the new one has not yet overcome this problem entirely.
The Joint Parliamentary Committee makes 86 recommendations overall but the central thrust is that while the direction of the draft bill is broadly correct, the government still needs to make significant changes, particularly in making a number of provisions clearer.
The bill aims to takes powers which were previously exercised using often obscure legal provisions and put them onto a clearer footing.
But one of the most controversial provisions is a new power - to compel companies to keep Internet Connection Records (ICRs) for 12 months which detail the sites everyone visits on the web (although not what exact page).
Not enough has been done to explain what is involved, the committee says.
Law enforcement rather than spies are the chief sponsors of this provision and the committee gives them support for the overall concept (even adding that they could be expanded to identify people visiting websites which hold illegal material rather than just for identifying forms of communication).
However, it says that more needs to be done on the detail. In particular there are problems over defining ICRs, over the technical feasibility of their collection and storage as well as the costs involved. All of this may make it challenging to deliver them in the timescale envisaged.
Bulk Personal Datasets are another area of concern.
Clear justifications and operational cases for why the powers are required remain lacking, the committee says, also criticising government for being unwilling to provide much detail of what kinds of datasets would be involved.
Some might be obvious such as passport records but others - like health records - would be far more sensitive.
It is notable in the report that the committee tried to get an answer on whether records such as health data would be excluded but was not given one by government.
There is relatively little detail on the issue of encryption which has been of concern to many companies (notably Apple which submitted detailed evidence).
A report from the Science and Technology Committee raised concerns from industry about the potential impact of ambiguity in some provisions.
The indications are that the government wants to retain its right to seek unencrypted data even if it knows it may not always be possible to obtain it.
There are still some concerns from industry over what is meant when its said that companies need to take "practicable" steps to provide such data and what impact it has on end-to-end encryption systems.
Some companies also remain concerned they could be compelled to hack into their own products under equipment interference provisions.
The Joint Committee also calls for some significant changes to the way in which judges provide oversight over powers to increase independence.
On intelligence sharing, the Joint Committee also notably calls for it to be made illegal to get another country to spy on behalf of a UK agency or body when it does not have the authorisation to collect such information.
The legislation is often technical by necessity and so a lot of weight is placed on "codes of practice" which outline how exactly the powers should be used. The committee say these should be published along with the bill to provide greater transparency and clarity.
The Intelligence and Security Committee report earlier in the week focused more narrowly on intelligence agency powers but took a tougher line.
It criticised the lack of an over-arching privacy protection and also said that when challenged about what the power of bulk equipment interference actually involved, officials were unable to provide an answer.
The committee have rejected the idea of a sunset clause in which the powers would expire after five years instead calling for a joint committee to review the use of the powers as the five year point approached.
There were clearly differences of opinion within the committee and despite the huge amount of evidence it took, some still feel it may have had to rush.
One member, Lord Strasburger argued the government had not learnt the lesson of drafting a bill too broadly and said that essential information was missing that parliament needed in order to decide if the powers were necessary and proportionate.
The issue now is how far the Home Office takes on board some of the criticisms and whether there is enough time to do so - a final bill is due to be introduced soon so that it can be debated and passed by the end of the year.