The PropTech fund has spawned a series of experiments around planning, technology and public consultation that are well worth following. It’s great to see DLUCH investing in pilots and local government.
Many of these experiments have focused on using AI in analysing open text responses to Regulation 18 consultations – tech as delivered by CitizenLab, CommonPlace and TheFutureFox
I’ve been trialing one such system and it’s really rather good. In-fact, around 80% accurate up-to 1,500 responses using OpenAI. CommonPlace have gone done another rabbit hole too – using their own Natural Language Model. I’ve actually sat and read over 100 consultation responses then looked at what the AI generated using one of these platforms – without any sort of concern for the results.
The potential savings are hard to quantify but for an average consultation (c.1500 responses), the pilot bodies report that their current costs for a single major consultation equate to approximately £3k and a process delay of approximately 3 weeks. Using AI, this could be cut by 50%.
However, the aim is not to take the human out of the loop completely and to use this technology in an assistive manner. And there are some issues – for example: –
- The accuracy drops when the sample size becomes too big.
- There is still a cost associated with OpenAI.
- The models don’t work well if contributions contain spelling mistakes or individual answers are verbose.
- Time needs to be invested in compiling the segments of AI synthesis into a report.
- We haven’t quite yet discovered if there are any serious biases in the sytem;
- There is a risk that personal information will be sent to a sub-processor as it is contained within the body of a response.
All of the above is likely to get fixed as the technology matures but there is an elephant in the room and that concerns governance, sensible use and transparency. Our view is it is both impractical and nonsensical to have an “opt out” clause for consultees but that some sort of adjustment needs making to policies and terms.
Ask your legal teams but we think that adopting the following sentences will put you any your stakeholders in a more comfort zone:-
- By using this service, you agree that we may use artificial intelligence (AI) technologies to help analyse and interpret any free-text feedback that you provide.
- Typically, we will use AI where there are a large number of responses for when speed is of the essence. We will indicate where AI has been used to assist us in the synthesis of views within the body of official output or outcome reports.
- No personal information will be processed using AI and there is no automated decision-making. Machine summaries will be checked and cross-referenced by a human against individual contributions for the purposes of maintaining accuracy and reducing bias.
- If you choose to use AI in the formation of views that you present to us, we may be able to detect this and we reserve the right to classify these as machine responses which are ineligible for further consideration.