Posted on

Part V: Future Of Work

Future Disruptions and New Maladies – what the future may hold…..

  • An aging population without a plan to take care of our old and infirm.
  • AI and ML: Machines are getting smarter and more intelligent. AI seems to be able to do a lot of the cognitive jobs currently held by humans.
  • Automation: Think self driving cars, trucks and other vehicles; IVRs that perform the jobs of CSRs, chat bots etc.
  • Are humans a single planet species? Can we defy gravity at scale (mass transportation as opposed to a few infrequent transgressions into space) – Will this open up new frontiers to explore and eventually inhabit new worlds?
  • Will we discover other intelligent species/civilizations? Again new frontiers for engagement outside our current boundaries of work – think Star Ship Enterprise and its crew – “Space: the final frontier. These are the voyages of the Starship Enterprise. Its continuing mission: to explore strange new worlds; to seek out new life and new civilizations; to boldly go where no one has gone before”.

All of these can be looked at as either opportunities or threats – it is what we make of it.  Change is never comfortable, but depending on how we embrace and adapt to it may mean our survival or ultimately demise.

Our response to these changes may take the following shape –

Protectionist Policies – Have they ever worked?

When we domesticated animals, did humans that performed manual labor, resist the entry of animals? – no, instead they evolved to managing these animals and accomplishing more work than what they could do individually.

Did humans resist mechanization – yes, but it wasn’t a disruption that they could resist for long because industrialization came with a number of benefits: it raised living conditions, increased life expectancy and also provided a preponderance of additional time that could be productively used in the pursuit of other interests and goals.

So why do we believe that the same response will be successful for the next round of redefinition of work?

Innovation: Can we reimagine work that humans do?

Why do we assume assembly line work is actually adding value?

Humans were first domesticated by agriculture and forced to leave their hunter-gatherer life style. For work that needed more strength than a single human, we used animal power for tasks such as ploughing or extracting oil.

Then came automation – where we invented gears, pulleys and mechanical arms or wheels driven by motors and generators to perform the  work on our behalf.

Next came CNC machines which in addition to the mechanical aspect of doing the job, could also be programmed to do the work without human intervention.

The final frontier is AI – where an automaton (or a software program since not all realms of work are physical) could sense the environment and determine the best action in order to achieve a set of finitely defined goals

I think we are ok as long as “we” get to define the goals but there are larger fears in society that when these automatons/programs take over and set their own goals – why would any human be needed?

Is that the right question? I do not know … does anyone?

Can AI and Intelligent robots really be an ally instead of a threat?

True if we do not hang on to our traditional definition for work and instead look for different ways to create value.

Will be a possibility if an alien species attacks us and we collaborate with our own robots to ward off this threat.

May be necessary if humans are not suited to be a space faring race and we need robots to help us in this hostile environment.

We will need them when we are older and frail and younger humans do not have the inclination or interest to look after us.

If we want to succeed, we will need to change the game …and have a plan for our workforce to transition to this new definition of work.

 🙂

 

Posted on

Part III: Consequences Of Rampant Capitalism…

We have seen a rapid growth in various world economies over the last century. It has been especially pronounced after world war II. Although of late this growth is not equally shared across the population. This results in income inequality and a sub set of the population loses all hope for upward mobility.

This segment of population is completely discounted and does not value the democratic choice that they exercise in elections, and consequently they make their decisions based on either  unachievable promises or even protectionist and racist policies.

 

 

Once the value of the vote goes down, what you see is election of Incompetent or Callous leaders into the government, which further endangers the existence of democracy or capitalism.

 

 

 

Now don’t get me wrong, Capitalism is the only economic system we have seen work – all others like socialism, communism, dictatorship, fascism etc. have all failed the test of time. The question really is how to temper capitalism to not become its own biggest enemy and threaten its own (and democracy’s) survival.

 

 

Our next post is a plan to tackle this….

 

Posted on

Part IV: A Plan To Address The Maladies We Have Seen…

So what is the crux of the problem:

  • Capitalism promotes migration of jobs and work to the cheapest possible location;
  • Probably ethically right since it rewards the group that is the most desperate, but is unfair to the group that has a net outward migration of jobs
  • Results in massive job losses and desperation in one location while net influx of jobs and prosperity in another
  • Retraining and a repurpose of labor at source location is usually not attempted or isn’t very successful
  • The group losing the jobs is politically powerless to resist or prevent it

Let’s do a simple SWOT analysis of all the players involved :

Company:

Strength:  Capital; Agile Business and Production Processes

Weakness: Bad Political Reputation, and hence opponents try to fight this practice using protectionist/nationalist policies

Opportunity:  Find the optimal production cost to maximize profits and share holder returns

Threat:  Competition achieves a lower per unit cost of production and thus  loss of market share

Investors & Share Holders:

Strength: Capital

Weakness: Run towards the best returns, sometimes very short term focussed rather than long; i.e. support a steep discount rate

Opportunity:  Deploy capital that can be most productively used; Employ private equity/venture capital investment constructs to lock in capital to generate superior returns in the medium term instead of meagre short term gains.

Threat: Other investors generating better returns

Workers at the outbound location:

Strength: Political say at the local level

Weakness: No Capital to invest

Opportunity: Human capital that is free

Threat: A lower cost location comes up and takes on the production and jobs from them; Not trained in anything other than what they are currently doing

Workers at the inbound location:

Strength: Political say at the local level

Weakness: No Capital to invest

Opportunity: Human capital that is free

Threat: Another lower cost location comes up and takes over the production and jobs from them; May not have any training

Given the incentives, a capitalistic society gravitates towards specializing and optimizing labor, capital and production usually towards the detriment of its own workers but in favor of its investor class.

What I propose:

Change the game to utilize the strengths for each player – investors to provide capital to competitive projects, localities to use their resources to set up the right competitive projects using local resources and labor, executors that have a proven track record to implement the projects in a transparent and efficient manner, operators to run the venture profitably and an infrastructure exchange that removes friction from investment, project execution, operation and the ability to move investments in and out of these projects that compete for being a better investment.

Infrastructure Exchange Market

What?

An open market place for micro/ project level sponsorship.

Starts with a contractual agreement between sponsors/implementors/operators on initial project parameters

Proposal put up for bit to the investors – micro credit – or ordinary shares available to folks with a well defined investment profile

Types of projects accepted:

– wind farms

– solar farms

– energy storage pods

– efficiency projects – reduce energy consumption run rates

– later could move to any infrastructure project – build roads / bridges etc.

Why?

– a way to fund infrastructure from goal minded folks that believe in sponsoring a projects goals while making sure of decent returns on investments

Goals can be

– renewable energy

– create jobs

– reduce pollution

– build infrastructure

– competition drives performance

– easy in and out through trading in the infrastructure bank exchange

How?

– sponsors propose projects on exchange

– investors buy in interest; when project fully invested in – kicks off

– implementors run with it and complete execution

– hand over to operators that run and produce steady annuities

– project shares trade on exchange in terms of how well they are doing and can be traded

– all activities have a std set of metrics that are measured and published for full transparency

– create a path for projects to fail and be winded down

– create market on which mature projects performing as annuities can be traded and folks can move in and out of the projects

– should you create it as a bond structure – like infrastructure bonds backed by local/state government?

When?

– create exchange with functionality for each participant

– get investors to fund operations

– sign up sponsors / implementors / operators

Where?

– where there is the greatest need

– local governments struggling with creating jobs but have local resources that can be leveraged

– Examples of local resources

– Land

– Wind power

– Solar power

– Hydro power

Who?

Initial investors – those that fund the exchange

Implementors – those that set up the projects initially

Operators – those that run and maintain the projects – create local jobs;

Investors – who fund the projects

Sponsors – local governments/state that can provide resources (e.g. Land lease/ rent etc)

 

An exchange like the one described above will align the incentives between the workers and the investors to create a win-win situation for both….

We are currently working on a prototype for the platform….watch this space…

 

 

Posted on

Part II: In Defense Of Government

We find “The Government” to be a common punching bag for most folks – politicians and common citizens alike.

Our politicians rail against a corrupt and ineffective government pointing to us  the ills of big government and why the regulations imposed by the government are stifling our industry.

There are folks that talk about the deconstruction of an institution that has evolved over centuries of human development. It seems, everyone has an example of some egregious behavior that they use to justify painting the whole institution bad and in need for pruning. And the travesty of the situation is that there is no one standing on the other side defending this vital institution and its usefulness.

So let me take on the defense for the institution called the “Government”, which allows us to cooperate in very large numbers, prescribes and maintains a rule of law and is able to undertake projects at a scale that is impossible/unsustainable for an individual/family or a small group.

Governments came about when humans started gathering into communities. They were three primary reasons to create a government –

  • Establishing a common benchmark of behavior in society and establish conformance
  • Achieve scale where an individual/family or small group could not
  • Making outsize bets in pushing expertise in any domain – agriculture, industry or technology (for e.g. core science) where short or medium term benefits may not justify a rational private investment

Establishing the Rule of Law:

Humans felt the need to create rules for common behavior that all members within a community to adhere to. Anyone not adhering to these rules was given punishments or incarcerations. The institution developed as a checks and balances for keeping civil behavior.

Bringing the Benefits of Scale:

If you look historically, humans formed collectives and villages/towns/cities etc. to take advantage of the power of the collective. There were some projects/endeavors that were beyond the scope of an individual or a small group like a family to accomplish. Hence we humans invented the construct of “The Government” to allow us to cooperate in larger numbers. For e.g.  maintaining a military for offense or defense or build roads that are more than point to point connections and useful for the entire community.

Before we assign all the blame to the government and we dismantle it – we also should be willing to give up all the gains achieved by this institution.

Are we ready to give up on our military, or our highways, the internet, the GPS system, antibiotics and miracle drugs we have on the market. All of these are innovations that started as government projects and were then handed over to private enterprise.

 

I do not discount the criticism leveled by some that there are government agents that take advantage of their power, or even some individuals who are free riders on the rest of society. Yes, you can find bad apples in any group, but let’s not use these examples to discredit the institution and forgo the benefits of having a functioning and effective government.

 

Posted on

Org Design: Governance Vs. Delivery

Benefits of keeping governance and delivery functions together

Over the weekend, one of my ex-colleague reached out to me seeking advice on an org design question – should you keep  governance and best practices functions within a delivery organization?

My take: You can either keep governance functions within your best delivery unit or create a separate governance organization but it won’t be successful without assigning it some critical delivery responsibilities.

 

Here’s an example of what has worked in my professional experience:

At Medco, while running the BPM COE, I was tasked with creating a structure that could parallelize development. We had a massive transformation project, with a scale up target of almost 1000 developers at peak.

We had applications for various products – like mail order dispensing, point of sale adjudications, specialty pharmacy etc. These applications included common workflow capabilities like order processing, customer advocacy, therapeutic resource centers etc. These we chose to implement as frameworks. There were multiple scrum teams working on parallel development. We created a governance group – the Corporate Agile COE that was responsible for orchestrating application delivery across these framework dev groups (COE’s) and the application work station groups (BIACs).  In addition to governance, this group also had some critical enterprise framework delivery responsibilities like authentication and authorization, PHI data access controls, single sign on, personalization framework, and service bus client-server framework.

While governance is a full time job, without delivery responsibilities, it does not carry enough creds to be effective.

Why?

Architecture and Governance should not become an “Ivory Tower”: It needs to be grounded, practical and implementable; Incentives are aligned between delivery and governance, so governance principles are light, not onerous and implementable. And the best way to prove that a design is implementable is to give the person/team proposing it a chance to implement it using the same guidance; The aim for both architecture and governance should be to be simple, rational, elegant and not onerous so as to impact delivery.

Eat your own dog food – hence establish credibility when prescribing your solution: The group prescribing architecture principles is able to demonstrate through their own delivery that it works, hence establishing credibility when prescribing solutions.

Architecture not a scape goat for delivery: Other delivery groups cannot claim that the architecture is unimplementable and thus make the arch and governance group a scapegoat for failed deliveries.

 

 

 

Posted on

Virtualization – A Necessary Strategy For Any IT Exec

Enabling technologies (inventions) have brought about faster innovation – now that change is constant, we are all trying to outdo the last incremental change. IT Execs have to worry about faster speed to market, which has shrunk from months to now days and even minutes.

Remember the days when a project had to schedule hardware and software change – and if you missed it on your project plan, either you were running on borrowed capacity or running extremely crippled till hardware was ordered, arrived and was provisioned in the data center. Those days are gone; today, capacity on demand is the norm and “one click provisioning” is taken for granted.

This is the case with the large investment bank that I work for where we have our own flavor of cloud and on demand provisioning; even with with smaller enterprises that can use infrastructure cloud providers (AWS/Azure/Digital Ocean/Google etc.) to spin up containers and add capacity on demand.

Very recently while mentoring a nonprofit, I came across a situation where the founder of a dance studio was forced to work on her IT systems more than the organizations mission. Upon probing deeper, we discovered that while IT tools are excellent productivity drivers, a fragmented landscape of solutions is actually a bigger headache to manage than doing this work manually with a pen and paper. This was a problem of fragmented providers and needing a lot of IT savvy to merge and manage data from the owner and founder of the non profit.

We recommended  a virtualized and a consolidated software solution which was accepted enthusiastically. We ended up setting up a word press instance for her in a matter of hours. In fact my 7th grader son pitched in and set up the entire static site for her. Then one of our other volunteers added various plugins like mail chimp and class scheduling and campaign management to the solution.

This is true democratization of tech – it’s not just the monopoly of large orgs with an army of IT folks – anyone can experience this new paradigm and turbo charge their nonprofit/business.

Here’s a graphic that depicts the types of offerings in the market. The boxes in blue represent what is currently available.

Posted on

Why Is Innovation Necessary? A Model Based Approach To Drive Growth…

I came across an interesting economic growth model, while auditing Scott Page’s “Model Thinking” class at coursera. Even though the models he discussed are relevant to economic growth for countries and why certain countries grow vs. stagnate, I felt that this is very relevant to the investment decisions that technology leaders make every day.

Here’s a summary of the basic growth model Scott discusses – even though its a very simple model, it brings out the core concepts that you have to weigh when allocating your project investments between RTB (Run the Bank) and CTB (Change the bank).

The model is set up on an island that has workers and coconuts. You can create picking machines out of coconuts (investment) that helps increase your output. Machines are lost at the rate of depreciation. So formalizing –

Lt = Workers at time t

Mt= Machines at time t

Ot= Output in coconuts

Et = Coconuts consumed at time t

It = Number invested at time t

s = savings rate

d = depreciation rate

Assumption 1: Output is increasing and concave in labor and machines (capital)

Ot = √Lt * √Kt

Generalizing then –

Ot = Lt (1-β) * Ktβ

Assumption 2: Output is either consumed or invested

Ot = Et + It

Assumption 3: Machines can be built to increase output (using a concave function) but depreciate

Mt+1 = Mt + It – dMt

Assumption 4: Investment for the next period is output times the savings rate

  It+1 = Ot * s

Since the output is a concave function,  investment has diminishing returns on the output.
To achieve a long term equilibrium,  investment would need to equal depreciation i.e.

Investment (Ot * s) = depreciation (Mt* d)

Since Depreciation is linear and the output in our model is a concave function, you would expect to see the following mapping for these effects:

So at a certain stage, because adding more machines is not going to produce enough output to sustain loss from depreciation, growth stagnates. The irony of this growth Model is that growth stalls or stops as the effect of depreciation becomes greater than the output produced through additional investment in new machines.

Next let’s consider the Solow’s growth model, that allows us to introduce innovation into the mix and addresses how to overcome this growth stagnation.

Ot = At * Lt (1-β) * Ktβ

where

Ot = output at time t

At = Innovation constant

Lt  = Labor at time t

Kt =Capital invested at time t

Here the key investment choices that can be made by a senior stakeholder is how to divide up the investment between automation (driving up Kt) or game changing Innovation (At); and in my experience you are headed for stagnation or failure without carving out a portion of your budget to invest into innovation rather than just automation!

Here Kt would represent your RTB budget while a portion of CTB (elements dedicated to innovation) would essentially push the innovation constant higher. The key question you have to grapple with is the distribution between the two. And every organization has its own appetite towards making such division between these two types of funding. A few key factors driving such decisions are –

– Organization culture

– Level of competition within the industry

– Risk appetite for the senior executive team

– Whether you are a listed on a public exchange or privately held

Thoughts? Would love to hear your experience in budget allocations and the kinds of biases or preferences you have experienced. Please comment below or send me a direct email.

Posted on

Responsibilities of a Product Owner

It is increasingly apparent that technology has become a large driver of business strategy. By providing new channels and mechanisms to interact with its customers, clients, partners and suppliers, technology is a key part of an organization’s sustainable competitive advantage. So its imperative for a product owner to not just understand the business landscape but also understand how he or she can use technology components as key artifacts in differentiating their offering.

Functional Responsibilities:

There have been a number of blogs which have focused on requirements that are intuitable, usable, simple and efficient. So I will not cover this aspect here.

Non Functional Responsibilities:

Security:

I cannot say enough on making sure your product is safe and secure for your users and their data and this focus is not only to comply with regulations like HIPAA but to stay out of the front page of the Wall Street Journal. Given the high profile breaches we have seen, it is imperative that your product has a plan for the following:

  • Encrypted data stores
  • Multi factor user authentication
  • Allow only encrypted access to your app or website
  • Std DMZ based protection for your applications
  • Regular security patch management program, software upgrades and periodic vulnerability assessment including pen tests and adhering to the OWASP Top Ten list.

Business Environment and Model Resiliency:

As a product owner you should keep an eye on the business environment and your competition, so that your business or application is not upended by a competitor introducing a game changing innovation or the market changing to be unfavorable to your product.

When I was working for a mail order pharmacy, we were able to use our functional components to rearrange the value chain and disrupt the pharmacy fulfillment market. Mail order pharmacy is a volume business and we used benefit plan design, formulary design and volume discounts to drive our profitability. The model did not involve engagement with a patient unless doing a coverage review or a generic substitution. We had implemented digitization of the prescription (i.e. separating the cognitive and the fulfillment parts of the process) to be able to route this to the most efficient pharmacist and/or robotic pharmacy to fill. Our competition – retail pharmacies on the other hand were local and a pharmacist at these retail pharmacy would have a one on one relationship with the patient. When we introduced the TRC (Therapeutic Resource Center) model, we were able to use our routing components to our advantage. We created these centers that were focussed on a specific disease condition (oncology, cardio-vascular, diabetic, neuro-psych, specialty etc.) and were able to route our chronic and complex patients’ calls and fills to these centers. The engagement that we got with these patients was a lot better than retail pharmacies, and we were able to drive mail order penetration using our plan designs by proving to our clients that we were able to bend the healthcare cost curves.

So when you are designing your product, keep your design flexible, so you can adapt if the environment or your competitor’s strategy changes.

Configurability:

Ask for your product to be configurable rather than needing code releases every time there needs to be a change. An example is the regulatory framework we designed for our on-boarding platform. Since my investment bank operates in multiple jurisdictions and there are frequent changes in regulations for any single jurisdiction; it was very beneficial for us to use a regulatory program framework with the rules extracted out of the application; These rules are user modifiable with the right approval workflow. This has given us a huge advantage in being nimble with regulatory changes and not have to wait for IT release cycles and change windows.

Modularize and Encourage Plug And Play Components:

When you modularize your system – like lego blocks representing various functions, you have the ability to introduce a champion-challenger competition between these blocks. This allows for increasing product innovation through constant improvements because of competition between the modules.

There are new capabilities being added every day in ML (machine learning) and AI (artificial intelligence). You want to have a flexible architecture so that you can plug these modules in anytime to improve the learning and adaptability of your apps.

Data Processing Variety:

Always design your application to be able to handle a multitude of data streams with different types of variety, velocity and volume characteristics. What I mean by this is your functional modules should be able to handle both structured (db, partner systems etc.) and unstructured data (say social media feeds/blogs). Given the growth in IoT devices, make sure you are able to source data from these devices and crunch them realtime to have a much more agile and realtime responses for your users.

Portability:

Be cloud ready – i.e. a portable application that is able to shift its compute and storage resources at any time. This is key for scaling up or from a business continuity planning perspective. Say you lose that data center that hosts your application. Are you able to port your application seamlessly to the disaster recovery compute / storage resources?

Platform Agnostic:

Technology obsolescence is a huge risk. Design your systems using std patterns, interfaces and open source components.  If one component goes out of its useful life or technical support,  it can be replaced by an equivalent functional component that isn’t obsolescent -this is not a question of “if” but “when”.

Scalability:

Your product needs to be able to scale up if there is a sudden rush of customer interest. It is useful to have a capacity on demand model, where you can dial up or down based upon market conditions. A number of cloud hosting providers have an elastic compute model offering that you can leverage.  For e,g, this blog site is hosted on an elastic offering with the ability to scale up or down depending upon demand.

 

SDLC Responsibilities:

Honest Broker for all Stakeholders:

A product owner is usually responsible for prioritizing the requirements from various business units and stakeholders into a single product. He or she must be an honest broker managing the interests and prioritization across these stakeholders in a fair, transparent and honest manner.

Promote Agile Development Practices:

Adopting an agile development methodology allows for iterative delivery where the development team has the benefit of frequent and constructive feedback and able to adapt to a dynamic market with rapidly changing demands.

Define Success and Measure Achievement:

Be able to define the metrics measuring application functionality, scalability and delivery before commencement of the scrum sprint. Should also be able to measure, communicate and put plans in place to address missing any of these goals.

Posted on

IT Industry Evolution

Some History

As humans went from hunter gatherers to agriculture, they settled down in small villages and hamlets. Once agriculture took care of food production, people could focus on specializing their skills – like being a shoe maker or a barber or an iron smith. They then bartered their services for food and products that they themselves did not produce. Next came the advent of money which brought about efficiency in transactions over bartering. After that came the industrial revolution with a large scale focus on mechanization. This was the era where machines powered actions that a human or a group of humans could not do and helped improve the efficiency of a single worker working in a job shop. This human history is very well documented in the Sapiens by Yuval Noah Harari.

The next level of efficiency was achieved by moving from “a job shop” production unit to an assembly line. Henry Ford is credited with introducing this innovation which brought about standardization and higher output. As production moved to assembly lines, there was the need to measure and standardize each action station to improve quality and throughput.

As industry specialized and process engineering became a science, data collection to fine tune processes whether to increase production or to reduce the cash cycle for a firm became important. Hence we saw industries investing in computing technology & resources. Now the trend is to not just eke out efficiencies, but to collect data from the environment or market to influence strategies on how to take advantage of these changes in preferences of consumers and to some extent even shape or mold user preferences.

How has the IT Industry Evolved?

Trends in Computing

Mainframe and Monolithic Architectures:

The first computers were large scale machines with very limited computing power. These mainly were invented as an academic project which later found application in data crunching within industry. Initially only the largest industries could afford it, and run it. I am sure you would remember the punch card drives and the huge cooling towers around the mainframes!

Evolution of Distributed Systems & Client Server architecture:

Since the 80’s we have seen an explosion in the speed at which information can be collected, processed and disseminated. Some advances came from the mainstreaming of IT into every business or organization. Initially the target was automation – simple enterprise systems that were automated for e.g. The production, planning, accounting, sales etc. There were a number of different flavors of distributed platforms that had specific appeals to different sets of users – the windows platform that was very popular in business user computing segment, the mac platform that appealed to an individual user with needs for creative art applications and the unix/linux platforms which appealed to the geeks. Eventually as we saw these platforms compete we saw the linux/java stack start to dominate the back end processing at most business enterprises, while the front end remained windows based and Apple made a big dent into the personal computing segment.

These led to centralized databases which facilitated the collection and analysis of historical data to make processes more efficient. A number of database structures evolved from network to hierarchical, then relational and object databases evolved as a result.

Web Infrastructure and Interconnection of computers

In the 90’s we had the development of the world wide web, with computers forming a connected web, with standardized communication protocols like TCP/IP, HTTP, SMTP etc.  This was a huge improvement to the unconnected islands that businesses and users had maintained prior to this. This really improved the velocity of information travel – from copying data to floppy drives and moving from computer to computer, to directly transmitting information from one computer to another when every node became addressable and ready to understand communication over standard protocols.

Development of Cloud Infrastructure:

In the 2000’s the trend moved to virtualization and the ability to run multiple processing slices on any physical machine. Computing became fungible and transferable. The idea was if every one was running their own physical servers which were not highly utilized, it would be better to have highly fungible compute and storage slices that could move around virtually to the least busy node, thereby improving efficiency multi folds for our computing and storage hardware.

IoT (Internet of Things)

We have also seen the shrinking of compute hardware to such an extent that this hardware can be embedded in any device or appliance. So a washing machine or a refrigerator may have enough and more computing power as a specialized computer from a few years back, implies each of these devices are capable of producing process data that can be collected and analyzed to measure efficiency or even proactively predict failures or predict trends.

Decentralized Value Exchange:

A parallel development has been the  paradigm shift in creating the ability to transfer value instead of just information. This came about from a seminal paper by Satoshi Nakamoto and the origins of block chain (more on this in another post).

Some call it a fundamental shift in philosophy where we do not need to depend on a central store of value (or authority) to establish the truth. It tackles the double spend problem in a unique and novel way without relying on this central agent. This has spawned applications in various sphere’s like digital currency, money transfer, smart contracts etc. that will definitely change the way we do business.

Big Data Computing

Today the amount of data has become so overwhelming that we find traditional centralized database architectures unable to keep up. This has resulted in a number of new architectures like Hadoop with its own map reduce algorithms and its family of peripheral components/applications like HDFS (Hadoop Distributed File System), Hive (interpreter that turns sql into MR code), PIG (scripting language which gets turned into MR jobs), Impala (sql queries for data in an HDFS cluster), Sqoop (convert data from traditional relational DB into an HDFS cluster), Flume (injesting data generated from source or external systems to put on the HDFS cluster), HBase (realtime DB built on top of HDFS), Hue (graphical front end to the cluster), Oozie (workflow tool), Mahout (machine learning library) etc.

Given this explosion of data that is being produced, collected, processed and acted upon which is beyond human capabilities, we have had to resort to delegating this to machines to collect, process and make sense of these trends – hence the renewed focus on machine learning and artificial intelligence.

Data Analytics and Usage of various Modeling Tools

Given the volume, velocity and variety of data we deal with, its humanly not possible for humans or organizations to analyze this steady stream of data and make sense of it. Hence we have started to construct models to analyze and make sense of this. There are a number of tools available that visualize and provide insights into this data and they inherently use a best fit model that is able to fit to existing data as well as provides predictive value to extrapolations of the causative variables.

This has become so ubiquitous in our lives that everything from credit scores to teacher ratings to how an employee is rated at work all depend on models. One of the critical insights in all this is that our inherent biases get encoded into these models. Hence we need to be careful to not trust these models without establishing their fairness. The best defense that we can employ is transparency and negative feedback control loops that help correct these models for accuracy. Cathy O’Neil has analyzed this very phenomenon in her book Weapons of Math Destruction – a delightful read!

Artificial Intelligence and Machine Learning:

Artificial intelligence is intelligence exhibited by machines. AI attempts to solve the following categories of problems using various methods (statistics, computational intelligence, machine learning or traditional symbolic AI) to achieve goals like social intelligence, creativity and general intelligence.

 

  • Reasoning
  • Knowledge Representation
  • Planning
  • Learning
  • Natural Language Processing
  • Perception
  • Robotics

There have been a number of efforts into machine autonomous behavior for e.g. autonomous driving cars, that use a variety of sensors like cameras and radars to collect information about the road and other vehicles and make real time decisions about control of the car.

 

Machine Learning problems can be broadly divided into supervised learning (where we have a body of data that a computer can use to learn and mimic), unsupervised learning (where the computer tries to make sense of patterns in a sea of random data whether through classification or clustering) and reinforcement learning (when a program interacts with a dynamic environment to perform towards a certain goal without an explicit teacher). A general categorization of machine learning tasks are as follows – classification, regression, clustering, density estimation and dimensionality reduction.

Is the ultimate design the creation of a self aware machine that can compete with humans for survival? Will this be a symbiotic relationship or a competition for survival? Are we simply a tool in the evolutionary process playing our part in creating a smarter, better and more resilient new being?

Posted on

Fake News and Information Velocity

Trends in Information Dissemination: Speed of Information Transmission is increasing at a very rapid pace and veracity (the source, the “Truthiness” of the information (thank you Stephen Colbert!), or if it has been tampered with en-route) has not .

Initially information travelled as fast as humans could travel – A courier would take a document or a piece of information and use a fast horse or chariot to convey it from one location to another. Then humans invented other modes of transmission of signals – carrier pigeons, smoke signals, sound/light transmissions within line of sight and then we moved on to using an electric current (telegraph, telephone) and eventually using light or or radio or microwave, VHF, UHF transmissions. We also learnt how to use satellites for communication when the curvature of the earth’s surface got in the way of line of sight transmission. Mostly this was point to point with respected agents in the middle establishing the veracity of the information – like newspapers, radio stations, tv stations etc.

Meanwhile in the early part of this century, we had the advent of a number of social networks like Facebook, LinkedIn, Twitter, SnapChat, YouTube, MySpace  etc. These networks connected individuals into social networks and facilitated an extreme agility in information transmission. Think  how easy it is to share an update or a personal opinion on these networks. It has developed to such an extent that it is possible to rapidly disseminate false news with real consequences.

Now it has become possible for anyone to act as an agent and share any content however authentic or inauthentic with his network. In a highly interconnected social network, this results in sacrificing veracity.

Take for example this specific site – focusnews.us, that very often shows up on my Facebook feed.

This site usually runs incendiary stories with provocative headlines like the following (I have only attached news items that showed up on my Facebook feed today because someone in my network liked it, or shared it or commented on it). Here’s a commentary of someone’s personal opinion disguised as news and sensational clickbait:

 

And having a few catalysts in any social network – implies a news story can be disseminated very easily without the usual veracity requirements as a traditional news organization.

The effect of rapid information dissemination and network amplification that helps false news propagate and have a much more profound effect on public perception.

I looked at their website to understand if they had any journalistic credentials. I found none – no names for the management team, the editor or any journalistic team.  The Contact Us section provided an anonymous email address (world@focusnews.us) that did nothing to reassure me of their credentials. So I am assuming this is a clickbait operation, with a goal to generate ad revenues by posting sensational stories on social networks and getting people to click on them. But I haven’t seen any ads on their site – so what should I assume? Is this a disinformation operation that is being fed to a gullible american electorate who loves their conspiracy theories!

When I looked up the internet registration record for this site, it appears to be registered in Macedonia and regularly publishing unverified US news.

RAW WHOIS DATA

Domain Name:                                 FOCUSNEWS.US

Domain ID:                                   D55884931-US

Sponsoring Registrar:                        GODADDY.COM, INC.

Sponsoring Registrar IANA ID:                146

Registrar URL (registration services):       whois.godaddy.com

Domain Status:                               clientDeleteProhibited

Domain Status:                               clientRenewProhibited

Domain Status:                               clientTransferProhibited

Domain Status:                               clientUpdateProhibited

Variant:                                     FOCUSNEWS.US

Registrant ID:                               CR255879236

Registrant Name:                             ivan stankovic

Registrant Address1:                         JNA 18

Registrant City:                             kumanovo

Registrant State/Province:                   macedonia

Registrant Postal Code:                      1300

Registrant Country:                          MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF

Registrant Country Code:                     MK

Registrant Phone Number:                     xxxxxxxxxxxxxxxxxxxx

Registrant Email:                             @gmail.com

Registrant Application Purpose:              P3

Registrant Nexus Category:                   C11

Administrative Contact ID:                   CR255879238

Administrative Contact Name:                 ivan stankovic

Administrative Contact Address1:             JNA 18

Administrative Contact City:                 kumanovo

Administrative Contact State/Province:       macedonia

Administrative Contact Postal Code:          1300

Administrative Contact Country:              MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF

Administrative Contact Country Code:         MK

Administrative Contact Phone Number:         xxxxxxxxxxxxxxxxxx

Administrative Contact Email:                 @gmail.com

Administrative Application Purpose:          P3

Administrative Nexus Category:               C11

Billing Contact ID:                          CR255879239

Billing Contact Name:                        ivan stankovic

Billing Contact Address1:                    JNA 18

Billing Contact City:                        kumanovo

Billing Contact State/Province:              macedonia

Billing Contact Postal Code:                 1300

Billing Contact Country:                     MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF

Billing Contact Country Code:                MK

Billing Contact Phone Number:                xxxxxxxxxxxxxx

Billing Contact Email:                        @gmail.com

Billing Application Purpose:                 P3

Billing Nexus Category:                      C11

Technical Contact ID:                        CR255879237

Technical Contact Name:                      ivan stankovic

Technical Contact Address1:                  JNA 18

Technical Contact City:                      kumanovo

Technical Contact State/Province:            macedonia

Technical Contact Postal Code:               1300

Technical Contact Country:                   MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF

Technical Contact Country Code:              MK

Technical Contact Phone Number:              xxxxxxxxxxxxxxxx

Technical Contact Email:                      @gmail.com

Technical Application Purpose:               P3

Technical Nexus Category:                    C11

Name Server:                                 NS1.FOCUSNEWS.US

Name Server:                                 NS2.FOCUSNEWS.US

Created by Registrar:                        GODADDY.COM, INC.

Last Updated by Registrar:                   GODADDY.COM, INC.

Domain Registration Date:                    Fri Oct 21 22:24:17 GMT 2016

Domain Expiration Date:                      Fri Oct 20 23:59:59 GMT 2017

Domain Last Updated Date:                    Mon Oct 31 15:37:59 GMT 2016

DNSSEC:                                      false

 

 

If people are gullible enough to fall for this clickbait and generate traffic for this website, there is nothing wrong with this – this lets them push advertising based on site traffic and a couple of kids making money off this,it is a perfect example of capitalism.

And who knows if they are also being paid by any foreign government to carry out a disinformation campaign – oh wait – thats not true – right – we just heard from Putin that they would never do so…so obviously this line of reasoning is wrong…

And if my fellow Americans are gullible enough to fall for these clickbaits and provide these entrepreneurs with a steady revenue stream – so be it. But let’s not call it news anymore….

 

Do also check out this report on how fake news works (from media matters.org) –