Smart Home, IoT, Data and Analytics – Reading List

Smart homes and the Internet of Things
A smart home is one in which the various electric and electronic appliances are wired up to a central control system so they can either be switched on and off at certain times. Most homes already have a certain amount of “smartness” because many appliances already contain built-in sensors or electronic controllers. Virtually all modern washing machines have programmers that make them follow a distinct series of washes, rinses, and spins depending on how you set their various dials and knobs when you first switch on. If you have a natural-gas-powered central heating system, most likely you also have a thermostat on the wall that switches it on and off according to the room temperature, or an electronic programmer that activates it at certain times of day whether or not you’re in the house.
https://www.explainthatstuff.com/smart-home-automation.html

8 sensors to help you create a smart home
List of sensors that one can deploy in their home to help ensure that they are safe from the storm.
https://www.ibm.com/blogs/internet-of-things/sensors-smart-home/

Six Evolving Business Models for the Smart Home
Smart companies today understand that traditional models of one-time sell through of products, or standalone products that cannot be monitored intelligently, provide little or no opportunity to create a relationship with consumers and establish brand loyalty. They also realize that in today’s market, their products can serve as revenue generators with the data they gather based upon a homeowner’s energy usage, security connectivity and/or home entertainment preferences.
https://www.iotforall.com/business-models-for-smart-home/

The anatomy of an IoT solution: Oil, data and the humble washing machine
A lot of people think about data as the new gold, but a better analogy is data is the new oil. When oil comes out of the ground it is raw, it has intrinsic value but until that oil is refined into petrol and diesel, its true value is not gained. Data from sensors is very similar to oil. The data that comes from the sensor is raw, to gain insight from it, the data needs to be refined. Refining the data is at the heart of a successful Internet of Things project which leads to business growth and transformation.
https://www.ibm.com/blogs/internet-of-things/washing-iot-solution/

The best smart home systems: Top ecosystems explained
Apple, Google and Amazon are the major players in the smart home arena now, with their smart speakers, ecosystems and voice assistants on hand to not only make controlling your connected tech easier, but to make home automation a doddle.
https://www.the-ambient.com/guides/smart-home-ecosystems-152

How data analytics is adding value in the smart home?
Analytics can be expected to foster whole-home integration of various Internet of Things devices by increasing awareness across multiple facets of the home, from thermostats to door locks to refrigerators to solar panels. Having insight across the entire home can enable machine learning and artificial intelligence technologies to eventually create smarter, more intuitive systems that not only make consumers’ lives more convenient, but also play a role in smart cities and the larger digitalized grid.
https://www.smartcitiesdive.com/news/how-data-analytics-is-adding-value-in-the-smart-home/446406/

Navigating Smart Home Data Security Concerns
Smart homes are no longer science fiction. But many consumers are slow to adopt. They’re worried about smart home data security breaches—and rightly so.
https://www.iotforall.com/smart-home-data-security/

User Perceptions of Smart Home IoT Privacy
Smart home Internet of Things (IoT) devices are rapidly increasing in popularity, with more households including Internet-connected devices that continuously monitor user activities. Report that analyzes the smart home owners reasons for purchasing IoT devices, their perceptions of smart home privacy risks, and actions taken to protect their privacy from those external to the home who create, manage, track, or regulate IoT devices and/or their data.
https://arxiv.org/pdf/1802.08182.pdf

Advertisements

Getting Started with Serverless Architecture

technology-1587673_960_720

Serverless Architecture is relatively very new. I’ve been exploring Serverless architecture for the new platform architecture off late. Though it is very interesting obviously there is a reasonable learning curve and I don’t see lot of best practices out there yet.

Everything looks green on the other side.. We will learn as we move forward..

Since, we use AWS as our cloud provider, most of the examples you will see are related to AWS Lambda.

Specific Reasons for exploring Serverless Architecture 

  1. No operating systems to choose, secure, patch, or manage.
  2. No servers to right size, monitor, or scale out.
  3. No risk to your cost by over-provisioning.
  4. No risk to your performance by under-provisioning.

https://d0.awsstatic.com/whitepapers/AWS_Serverless_Multi-Tier_Architectures.pdf

One thing i learnt in the last few years about developing distributed applications is that, it is not about learning new things… it is always about unlearning what you have done in the past.

If you are specific about Vendor lock-in then this may not be a choice at all for you…

Following is my reading list on Serverless Architecture.

What is Serverless?
https://auth0.com/blog/what-is-serverless/

Serverless Architectures
http://martinfowler.com/articles/serverless.html

What is Serverless Computing and Why is it Important?
https://www.iron.io/what-is-serverless-computing/

Serverless Architecture in short
https://specify.io/concepts/serverless-architecture

Is “Serverless” architecture just a finely-grained rebranding of PaaS?
http://www.ben-morris.com/is-serverless-architecture-just-a-finely-grained-rebranding-of-paas/

IAAS, PAAS, Serverless.
https://read.acloud.guru/iaas-paas-serverless-the-next-big-deal-in-cloud-computing-34b8198c98a2#.m9us1c5fe

Serverless Delivery: Architecture
https://stelligent.com/2016/03/17/serverless-delivery-architecture-part-1/

Principles of Serverless Architectures
There are five principles of serverless architecture that describe how an ideal serverless system should be built. Use these principles to help guide your decisions when you create serverless architecture.
1. Use a compute service to execute code on demand (no servers)
2. Write single-purpose stateless functions
3. Design push-based, event-driven pipelines
4. Create thicker, more powerful front ends
5. Embrace third-party services
https://dzone.com/articles/serverless-architectures-on-aws

Serverless Architectures – Building a Serverless system to solve a problem
https://serverless.zone/serverless-architectures-9e23af71097a#.j9z60nxw1

Serverless architecture: Driving toward autonomous operations
https://www.slalom.com/thinking/serverless-architecture

Serverless Developers
https://serverless-developers.com/

The essential guide to Serverless technologies and architectures
http://techbeacon.com/essential-guide-serverless-technologies-architectures

Using AWS Lambda and API Gateway to create a serverless schedule
https://www.import.io/post/using-amazon-lambda-and-api-gateway/

Five Reasons to Consider Amazon API Gateway for Your Next Microservices Project
http://thenewstack.io/five-reasons-to-consider-amazon-api-gateway-for-your-next-microservices-project/

AWS Lambda and the Evolution of the Cloud
https://blog.fugue.co/2016-01-31-aws-lambda-and-the-evolution-of-the-cloud.html

SquirrelBin: A Serverless Microservice Using AWS Lambda
https://aws.amazon.com/blogs/compute/the-squirrelbin-architecture-a-serverless-microservice-using-aws-lambda/

A Crash Course in Amazon Serverless Architecture
http://cloudacademy.com/blog/amazon-serverless-api-gateway-lambda-cloudfront-s3/
­
AWS Lambda and Endless Serverless Possibilities
https://abhishek-tiwari.com/post/aws-lambda-and-endless-serverless-possibilities

Awesome Serverless – A Curated List
https://github.com/JustServerless/awesome-serverless

Happy Learning!

Developing a Robust Data Platform : Key Considerations

key-considerations

Developing a robust data platform requires definitely more than HDFS, Hive, Sqoop and Pig. Today there is a real need for bringing data and compute as close as possible. More and more requirements are forcing us to deal with high-throughput/low-latency scenarios. Thanks to in-memory solutions, things definitely seems possible right now.

One of the lesson I have learnt in the last few years is that it is hard to resist developing your own technology infrastructure while developing a platform infrastructure. It is always important to remind ourselves that we are here to build solutions and not technology infrastructure.

Some of the key questions that needs to be considered while embarking on such journey is that

  1. How do we handle the ever growing volume of data (Data Repository)?
  2. How do we deal with the growing variety of data (Polyglot Persistence)?
  3. How do we ingest large volumes of data as we start growing (Ingestion Pipelines/Write Efficient)?
  4. How do we scale in-terms of faster data retrieval so that the Analytics engine can provide something meaningful at a decent pace?
  5. How do we deal with the need for Interactive Analytics with a large dataset?
  6. How do we keep our cost per terabyte low while taking care of our platform growth?
  7. How do we move data securely between on premise infrastructure to cloud infrastructure?
  8. How do we handle data governance, data lineage, data quality?
  9. What kind of monitoring infrastructure that would be required to support distributed processing?
  10. How do we model metadata so that we can address domain specific problems?
  11. How do we test this infrastructure? What kind of automation is required?
  12. How do we create a service delivery platform for build and deployment?

One of the challenges I am seeing right now is that the urge to use multiple technologies to solve similar problems.  Though this gives my developers the edge to do things differently/efficiently, from a platform perspective this would increase the total cost of operations.

  1. How do we support our customers in production?
  2. How can we make the life our operations teams better?
  3. How do we take care of reliability, durability, scalability, extensibility and Maintainability of this platform?

Will talk about the data repository and possible choices in the next post.

Happy Learning!