Hello đź‘‹ Welcome to my corner of the internet. I write here about the different challenges I encounter, and the projects I work on. Find out more about me.
After working together with dozens of real estate developers in the Netherlands, over the course of more than a year we identified a problem that required turning texts into spatial attributes. The texts were millions of pages of similarly phrased but differently composed lines of legal jargon.
For us founders, both fundamentally without AI background, the parameters of this problem were:
These parameters indicated a product opportunity, possibly well-suited to solving using AI.
De-risking this product opportunity for us then required answering:
We could answer these questions through our network: we contracted a feasibility study by an NLP-specialised engineer that was in our direct network, and we explored whether we could access more AI-talent. We could. The feasibility study outlined the possible techniques, the approximate steps from nothing to MVP and the required time per step.
After consideration, we decided to proceed, and hired the engineer part-time, together with two junior AI engineers to support him.
Product development took over a year, and while the resulting solution bore some fruits, it eventually proved insufficient to properly grow revenue, and also insufficient to raise additional funds for continued development. In my eyes, the following three oversights were decisive towards this result.
The value of the eventual product lied in the ability to reliably combine multiple (typically 10-50) sentences of legal jargon that outline the spatial allowances for a given parcel, and harmonise these rules into one maximum buildable building envelope. Comparing this to the actually observed reality, would yield the parcels that allow immediate construction; valuable information, especially in a tightly zoned country like the Netherlands, where every buildable square meter is typically utilised.
However, the most interesting cases would prove to be relatively small set of parcels parcels that had rather particular and unique rules.
The fact that a small amount of rather unique cases provide the majority of the value, created certain requirements of the system, that in the end turned out to be prohibitively expensive to develop.
Most of all, it required an extremely high accuracy. An accuracy of 90-95% may be attainable (and we did attain that for a specific typology), and 99% certainly is as well, especially in a homologous country as the Netherlands, but absolutely speaking it still leaves enormous amounts of parcels out-of-view, to be checked manually. The Netherlands contains approximately 19 million built structures. A 99% accuracy still leaves 190.000 structures to be manually reviewed, which is definitely unrealistic.
💡 Learning: founder-domain fit needs to be extremely strong in that there needs to be deep understanding of exactly where the value is created in a process, and they need to be able to articulate the “boundaries” to providing that value, in absolute terms. 99% was insufficiently accurate, in our case.
The initial approach was NLP (Natural Language Processing) based, because it concerned a case of “a computer needing to understand human text”. The feasibility study investigated that approach. Yet, during development, existing NLP tech proved insufficiënt, and the system “devolved” to a rule-based approach. Transformer-based approaches were not seriously explored.
The oversight here lies in multiple aspects. Firstly, our specialised engineer saw opportunities in their own niche (NLP), versus a wider lens. Secondly, we decided to trust in the engineer, as we did not have our own footing in the technologies. Without us having the knowledge, we put too much faith in the hands of the engineer. Lastly, we did not offset this lack of know-how with gate-points: clear go/no-go moments based on required accuracy vs. maximum achievable result through our solution.
The reality therefore was that there was an insufficient problem-tech-solvability fit, and we lacked the (typically self-created) accuracy figures to make hard upon. As a result, we permitted gradual development with a well intentioned team, that eventually led to a product that simply missed the mark.
đź’ˇ Learning: in a tech dependent product/solution, knowledge of the maximum capabilities of the tech need to be part of the founding team
We chose to initially hire a part-time team: 0.2 to 0.6 FTE senior engineers, and some dozen junior engineer hours per week. The slow(er) rate of development bogged down the general decision making process as it created an “excuse” for absent results through “lack of hours”. In reality, the two crucial factors were not only lack of hours, but also an unsuitable tech solution. Hiring employees full-time employees implies more financial stress, more requirements for the product to work, and therefore less tolerance to excuses. I am convinced that one month full time would have exposed the problems we now realised over a year, yet would have cost a fraction of what the full process costed in our case.
đź’ˇ Learning: going hard creates an environment with less tolerance to excuses and lack of results