Hi @peter! There's a lot going on in this thread, but I'll try to address the main questions and ideas:
- BTW minor detail: there's actually no need to store an API key in the app. Instead you would generate an Ethereum private key (hex string of 64 characters) for the user and store that locally. No shared secrets are needed, and the user will authenticate by signing a challenge in the challenge-response authentication protocol. The private key will also enable the user to access their share of the earnings. We'll provide more details when the Community Products dev docs become available.
write-only permissions to streams are actually possible. Such permissions can't be created via the Core UI (it will always grant
read+write, but if I remember correctly write-only permissions can actually be granted via the API, as every permission appears as a separate permission object. I don't think anyone's used this yet, but as you pointed out it might become important for Community Products use cases, so we'll take a look at it as part of establishing Community Products best practices and features.
- We can't use oAuth because that's built for centralized services with centralized gatekeepers. It would be ok right now but not in the long term, as we move towards decentralization. Instead of centralized authentication, we must use encryption to protect data.
- How you model your streams and products is up to you - whatever best fits your use case. If you have multiple types of data I recommend having a stream for each one of those. This approach would also give the streams a stable structure/schema (i.e. all messages in a stream would have the same structure). Then you can add these streams to one or more products as you wish. This way you can control access to different data content as granularly as you like.
If you have 26 different data collectors, you could have a stream for each (for granularity when needed).
Additionally you could duplicate similarly-shaped data together, for example have a stream with "Search Query" data across all the different search engines. Remember that you can easily send data to multiple relevant streams. So when a user searches on Google, you could send the event to "Google Search Queries" and "All Search Queries" streams. This might make your product easier to use for some buyers.
Then you can add these streams to products as you wish - this means deciding your product packaging. You could have several products with different content, like "Everything", "Google only", "Search queries only", etc. Buying a particular product would of course grant access only to the streams contained in the product.