So you are a software developer who loves working on creating crisp interfaces for customers, or you are a hobbyist explorer learning about frontends, you would have definitely crashed upon the jargon called Micro-frontend. We will explore what the fuss is all about.

Micro-frontends (MFE) in essence are just coded repositories built using the same technology that we use to create the monolithic frontend, except, MFEs can communicate with other repositories, depending on the structure and design, either unidirectional or bidirectional.

There is a host or orchestrator that holds up the main application, which can hold the micro frontend repositories. Y’all know about IFrames, and how they can hold a specific page or web view in it, think of MFEs in the same fashion, but in a broader sense.

Okay but why?

Well, imagine having an application full of various full-blown features. And imagine the intricacies if the application has role-based accesses, or polymorphic view for the pages, or the subset of modules, and that it can have more roadmaps in the future. Suppose, for example, Product Manager (PM) comes up with a customer journey, let's call it PiramalCommerce, which initially had a self-KYC, a registration, and an application creation section.

But in the future, the Product Managers at PiramalCommerce planned to bring a payment gateway, an OKYC/CKYC system, a completely loan-variant-based experience, and a full-blown in-app support experience. While the developers can make the decision to make it all within a single application, thus saving time for the product, but if the need arises for the other verticals of Piramal to bring in a gateway, and in-app support portal, the developers would have to access the repository, read the code, CTRL+C, CTRL+V reusable parts of it.

Well, if the subproduct is reusable to such a good extent, for ex. payment gateways, can’t it be leveraged to a whole extent and thus make a micro frontend out of it? This same platform will be used by other teams in the future to expand the features and reuse the same codebase.

Micro-Frontends Architecture consists of 3 main components:

1. Orchestrator (Piramal Commerce)
2. Micro frontend (Payment Gateway)
3. Framework (a library that lets the applications communicate, ex : Module Federation)

Still why MFEs?

While MFEs help development in many ways, there are several benefits to why use them. Scaling a project becomes difficult as the code starts to pile up, and it becomes tricky to maintain such a bulk amount of code, micro frontends help to reduce the dependency. Here are some of the benefits of using MFEs.

1. Single Responsibility
2. Don’t Repeat Yourself
3. Reusability
4. Agnosticism of technology
5. Open-Closed Principle
6. Platform/Domain Driven Development
7. Flexibility

Overall, micro frontends can help to make frontend development more efficient and effective by allowing teams to work on smaller, more manageable pieces of the application.

There are a few situations where using micro frontends might not be the best approach:

1. Small applications: If you are building a small frontend application, the overhead of implementing micro frontends might not be justified.
2. Tightly coupled features: If different parts of the frontend application are closely interconnected and depend on each other, it might be difficult to divide the application into smaller pieces without introducing complexity.
3. Lack of developer resources: Implementing micro frontends requires a certain level of developer expertise and resources. If your team does not have the necessary skills or resources, it might not be practical to use micro frontends.
4. Lack of support from frameworks or libraries: Some frameworks or libraries might not have built-in support for micro frontends, which can make it more challenging to implement this architecture.

Overall, whether or not to use micro frontends will depend on the specific needs and constraints of your project. It might be worth considering micro frontends if you are building a large, complex frontend application and have the necessary developer resources and support from frameworks and libraries.

How to use micro frontend architecture?

1. Identify the parts of the frontend application that would benefit from being divided into smaller pieces.
2. Decide on a way to divide the frontend application into micro frontends.
3. Set up the necessary infrastructure to host and deploy the micro frontends.
4. Implement the micro frontends using the chosen approach. This could involve using a library or framework to handle the communication between micro frontends, or building custom solutions to handle this communication.
5. Test and debug the micro frontends to ensure that they are working as expected.
6. Deploy the micro frontends to production. This might involve deploying each micro frontend to a separate domain or subdomain, or using a service worker or server-side rendering to combine the micro frontends into a single application.

Overall, the process of using micro frontends will depend on the specific needs and constraints of your project, as well as the tools and technologies you are using.

In conclusion, micro frontends are a powerful way to build and maintain large frontend applications. By breaking up the front end into smaller, more manageable pieces, teams can work concurrently, improve scalability, and enhance maintainability. As more and more organizations adopt micro frontends, we may see even more innovative and effective ways of using this architecture in the future.

References :
1. https://webpack.js.org/concepts/module-federation/
2. https://www.npmjs.com/package/module-federation-plugin

The WHAT, WHY and HOWs of Micro-frontends was originally published in Stories by Piramal Finance on Medium, where people are continuing the conversation by highlighting and responding to this story.

~ Motivator Bot

Did you ever miss a calendar event because you forgot to check it? Or did you miss a Github issue which could’ve been worked up on earlier only if you checked it ? Ever forgot doing something because you didn’t have reminder in place ? Or ever felt the need to have someone who wishes you happy things without any intention or greed ? 🥺

Ideation 🤔

Well, you are not alone, we are on the same page, so when I came to know about @Crio.do Crio #IBD Remote Working Tools edition, I thought why not to give my thoughts a go. I thought a chat bot could do many of the things that we can’t even imagine. Have used several telgram or discord bots, such as stock buy/sell signal bots, to mention the least. Why not make my own one for Crio #IBelieveinDoing edition? 👩🏻‍💻

I thought to make a bot which helps me/people like me in their day to day office work. I wanted to make it in a way that can be scaled up, and updates to it can be rolled out in an efficient way. Bot making has a lot of scope for learning and improvement, and something which takes time to make it robust. A bot that is usable and isn’t an annoyance. And a bot who is a good friend. 🤭

So which platform to write bot for ? There are lot of options, discord, telegram, flock and what not. But I thought, since I am well known to Slack UI and workspace since I am doing job in a firm that entirely uses it, and well Crio uses it as well, felt it would be a good try to explore the API documentation of slack. 👀

I wanted the bot to tackle the pain points faced by me :

a. Monday motivations and some good vibes daily. 😛
b. I shouldn’t forget simple things, and would like to have something that reminds me of what to do and when to do. ⏱
c. Don’t want to miss Google calendar events. I missed my first retrospective once and that’s something I dont wanna miss ever.🗓
d. Have something in place that alerts me when I am tagged or an issue is created or someone creates branch of a repository. ⚠️
e. A shortcut to locate the repositories, from slack itself because its else too much of redirection. 🚴🏻‍♀️

Planning 🧐

So I started traversing the Slack Web API, and Events API, could split the tasks into milestones of learning ( the Crio way aka Learn by Doing), such that

1. Learning how to create slack bot from the API
2. The authentication flow to use it in Javascript code
3. Identifying the permissions needed to perform specific actions (I dont wanna give unnecessary permissions to the bot which I can’t risk to be accountable for if someone else uses it)
4. Third Party Integrations (read tough cookie 🍪 ) research
5. API research and debugging raw calls in postman
6. Testing the code and modularizing, optimizing, identifying dead code and code smells. ( read sonarQube or Code Inspector docs, avoid tech debts(for long term) )

Specifics

So I created atleast 3 to 4 bots to understand what each different permission set causes the bot to behave. Looked a lot through API documents to understand use case of each permission set.

The tech stack focused on the following :
1. To send good greeting messages and reponses :
Identified the free APIs
https://api.adviceslip.com/advice
https://www.affirmations.dev
https://type.fit/api/quotes

2. Authentication Flow and Bot types :
Studied Auths, Legacy bot, New OAuth Bot, Differences between both, which one would be best to use. (I got stuck here for good amount of time 😅)

3. Identifying the permissions and functionalities I would need :
Events API, Event Subscription, Webhooks, Slash Commands, Socket mode : when to use and when not to, App manifest

4. Third Party Integration (read tough cookie ? )
For this I preferred using a middleware for the time being, Pipedream API came as a boon, could integrate a lot of things in it with the able to code at runtime inside it. Had some gotchas which are expected when you use such tools but it is a great middleware to integrate codes with third party. (Learn oAuth flows before diving in for middlewares for third party applications if you have plenty of time)

Studied Google Calendar API and Github API to see what all are the possibilities, to be honest, its endless but tried to implement the alerts that are most important and feasible to implement for time being.

Following are the Integration workflows that I created to integrate and run events/webhooks from Github to Slack, Webhook to Github and Calendar to Slack.

• Issue Creation : https://pipedream.com/@rprasad43255/github-to-slack-communication-issuecreation-p_PACWRBV
• New Branch : https://pipedream.com/@rprasad43255/github-to-slack-communication-new-branch-p_OKCYwGr
• User Mentions : https://pipedream.com/@rprasad43255/github-to-slack-communication-new-branch-p_OKCYwGr
• Search repo : https://pipedream.com/@rprasad43255/github-to-slack-communication-search-github-repo-p_PACWRJZ
• Get Next Google Calendar Event : https://pipedream.com/@rprasad43255/googlecalendaralerts-p_YyCWZwG

5. Understanding scheduling and using it in the system.
Gotchas and caveats around it.

6. Studying the documentation of libraries such as

• Slack Events API
• Slack Web API
• friendly cron
• node cron
• axios
• ngrok

So whats the working model :

A simple bot which pushes you forward to do more with the help of alerts, reminders, schedulers, and Github, Google Calendar Integrations.

• How many times do we miss or get late on checking Github issues, mentions, or branch alerts. No more now, using Pipedream API to integrate Github, now one can directly get notified when they are mentioned, or a branch is created, or issue is raised for a repo.
• Have you dozed off with naptime while a calendar event was scheduled for next one hour and so you missed it ? Say no further, you can now get alerts if there’s a meeting in an hour with help of Pipedream integration and some JS snippets.
• Want to quick search your favorite github repo stats, or get issue counts, no need to open chrome. Just /searchrepo it.
• In order to stay on track and to not miss out anything, one can set reminds for himself or for channel with precision of minutes or hours or days. Just remindme or remindus it
• Want to set scheduled messages with a friendly precision, you got it. Just schedule it
• Often during work all we feel is a need to find some positivity to deal with anxiety, stress or fear or even imposter syndrome. Sometimes words are the best friend we need. This bot posts a randomized good morning alert, advice at noon, and quotes in evening.

Crio-WFH/Slack-extensions

The screenshot highlights one of the supported keyword, help, which returns the possible functions that it can do. These 6 functionalities cover pretty much all what the bot is capable to do as of now.

Thats it, folks. Will upload more such blogs around new updates made to the bot. Thanks for reading. Have a good day. And clap please. 👏 🤭

#CrioIBDRemoteWorkingTools #IBD Remote Working Tools Crio #IBelieveinDoing #CrioIBD #IBelieveinDoing #LearnbyDoing #Criodo #IAmaBelieverofDoing

A poem based on a relationship that is beautiful, yet hurts like a flame. Based on a Twin Flame relationship, it’s drive and effect on the writer’s mind.

I don’t know what to write,
I don’t know if its wrong, or right.

I don’t know what to say,
is it like an insomniac night or a day?

A vibe is what I could feel,
a dagger in my heart,
yet there’s nothing that it can’t heal.

A flame is what it feels like,
beautiful and luminescent,
and warm yet scary delight.

A potent connection it feels like,
yet so impotent is the luck,
which I can’t describe.

It is like a star that I endear,
but also a scar,
that I can’t bear.

It’s like a flame,
too hot to handle,
yet it is bittersweet,
and my destiny’s a little scandal.

It’s an incandescent moon that I love,
but a monsoon that comes too late,
and goes off so soon.

It’s like a chase which seems a waste of time,
but every moment is worth it,
as in the end,
everything rhymes.

Ever wondered how a black hole would look like ? If yes, great.

But ever wondered how it would feel like to fall in it ? Obviously no, unless and until you tried to figure it out !

Even if you ever imagined, still you must hadn’t felt how it feels to fall in there. I would still consider myself lucky that I had such an experience, whilst sitting on a cubicle chair. Wondering how ?

It’s all the magic of the amazing world full of realities. I mean to say, its all possible due to the world of Virtual Reality, where everything is possible. Be it a space, or a rollercoaster ride, or a world full of horror, one can experience the thrill of everything, all at once, all credits to the AR and VR world.

You must be wondering what is VR or AR if you are from a non technical background. Even if so, most probably people are known of these terms, because this technology is itself known worldwide.

Virtual reality is a computer generated simulation of an environment that can be interacted within a real or a physical way by a person using a special equipment such as a headset with a screen embedded in it or a hand controller that helps to interact.

In simple non-technical words, VR is a technology that enables the users to experience the realistic world without being actually in it. AR, on the other hand simulates a model into the real world which can actually provide a composite view.

Mixed reality is the fusion of VR and AR altogether wherein the advantages of AR and VR both are incorporated in a single environment. Apart from these, there are other dimensions of VR, such as XR etc. WebVR platform too is developed for making VR scenarios accessible in a website.

Okay, so to be honest I didn’t knew much of VR till January ’19, but when I came across the problem statement of a furniture manufacturing company, i.e. Pepperfry during Smart India Hackathon 2019, it intrigued me. The problem statement was to make augmented models for the furniture that can be used for making tutorials which could make the people understand the assembling process of furniture.

I didn’t had enough skill set to make all those things and to render it in a 3D environment. But when I came across the google form that mentioned internship for VR, I couldn’t resist but apply for it. And that’s how I came to know of Sastra Deemed University and Sastra TBI, which laid forth the foundation of my VR journey.

The best thing about VR is that it’s applications are multidimensional. One may think that the scope of VR is only till the boundaries of Gaming and entertainment, but instead VR has its roots in almost every field, if one has a vision to do something, he/she can.

Be it biomedical, gaming, interior design, face mesh wrapping, shopping .etc, VR has played its vital role in every field. Examples such as Human Anatomical study, Pokemon GO, Furlenco. , Snapchat, Lenskart. etc are some of the best successful examples which employs the VR and AR, all in a nutshell.

The best thing about seeking an internship in Sastra TBI is that they have the sort of environment which one hopes for. Be it the fully rigged workstation, or a high performance Oculus Rift system, or a functional laboratory, everything is present in here.

We, the students of VR lab were assigned the task to work up on themes such as interior designing, VR pranks, heritage tourism using VR, Human Anatomy using VR and AR based furniture mapping. These five problem statements were distributed among 5 groups which are currently working up on it.

I, along with two group mates received the task of making VR Prank games wherein we are incorporating Horror Prank and Richie’s Plank. Almost 70% of work is covered but we are still looking up for more. During its implementation, we tried to understand and realize what are the problems one can face whilst being in a prank environment first, by putting ourselves in their shoes and then started working up on it to make things more realistic.

This project focuses on bringing new people who have never experienced VR experience the feel of it. And it’s thrilling to work up on it, too.

VR is more about user than the environment. The environment changes as per the user and vice versa, and also the developer has to take the co-ordination between them in check. Moreover, as much as I realized and understood, VR is much about physics concepts and creativity.

No matter what, VR is the technology that can influence so many lives in a way one can’t think of. And to be frank, it’ll grow rapidly as more and more people are exploring it’s domains.

And I feel more than lucky to be working up on it.

Virtual Reality ! was originally published in The Virtual Ink on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction: The D3 is an abbreviation of Data Driven Documents, and D3.js is a resource JavaScript library for managing documents based on data. D3 is one of the most effective framework to work on data visualization. It allows the developer to create dynamic, interactive data visualizations in the browser with the help of HTML, CSS and SVG. The data visualization is the representation of filtered data in the form of picture and graphics. Graphical or pictorial representations presenting even complex data sets with ease. Also, comparative analytics or patterns can be easily traced with the help of data visualization, thus enabling the client to make decisions without much brainstorming. Such visualizations can be easily developed using frameworks such as D3.js. Mike Bostock wrote the D3 framework. Before D3, Protovis toolkit was widely used for Data Visualizations. Though there are many other frameworks for Data visualization, but D3.js has left its footmark because of its flexibility and learnability.
Instead of working as a monolithic framework providing every conceivable feature, D3 solves the core of a problem by providing efficient manipulation of data. It reduces the overhead and allows flexibility.

Features: There are many other platforms or frameworks available for managing data visualization, but D3 has left all the other frameworks behind because of being extremely flexible. Following are the major features which separate D3 from other frameworks:

• As D3 uses the web standards such as HTML, CSS, SVG. It renders powerful visualization graphics.
• Data driven approach allows D3 to retrieve the data from different web nodes or servers and analyze it furthermore to render visualizations. Moreover, it can also use static data for processing.
• D3 allows variations in tools for creation of graphics. It is a basic structured table or a well analyzed pie chart. It’s libraries varies from the most basic tools to advanced set of resources. Even complex GIS Mapping can be done using D3. Even it allows the customize visualizations as per the need. Nevertheless, its all possible due to its support to Web Standards.
• It even supports large datasets and makes the most use of its predefined libraries, thus enabling the users to reuse code.
• Transitions and animations are supported and D3 manages the logic implicitly. Thus one doesn’t need to manage or create them explicitly. Animation rendering is responsive and supports fast transmission between internal states.
• One of the key feature of D3 is that it supports DOM manipulation and is flexible enough to dynamically manage the properties of it’s handlers.

Syntax:
D3 uses the JavaScript functions to carry out most of the selection, transition and data binding tasks. CSS also plays a key role in styling the components. Moreover, JavaScript functions can be scripted in such a way that they can read out data present in other formats.

• Selection: Before working on a dataset, the major task to be carried out is selection, i.e. retrieval of data from dataset. D3 enables the selection task by passing a predetermined tag as a parameter to the select function.
• Similarly, one can work up on various datasets defined under specific tags. Parameters to selectAll() can be tag, class, identifier or attribute. Elements can be modified or added or removed or manipulated and all this is based entirely on data.
• Transitions: Transitions can make the values and attributes for a dataset dynamic.
• In above scenario, notice that for all the elements coming as a subset of pre tag, transitioned accordingly.

For more advanced usages, D3 makes the use of loaded data for creation of objects and manipulation, attributes addition and transitioning is done accordingly. All these operations come under the Data Binding part.

Setting up D3.js environment: In order to make use of D3 for a website or webpage, first thing that needs to be taken care of is its installation or import of library into the webpage.

1. The D3 is an open source library. The source code is freely available on the D3.js website. Download the latest version of the library.(5.7.0 currently)

1. Download D3 library from the source link[/caption]
2. Unzip the .zip file which obtained after the completion of download. Locate the d3.min.js file which is the minimal version of D3 source code. Copy that file and include it in the root folder or the main library directory of web page. In webpage, include the d3.min.js file as shown.
3. Note: D3 doesn’t support Internet Explorer 8 or its lower versions. Preferably use Safari/ Mozilla Firefox or Chrome.
4. Example: The basic example shown below demonstrates the use of D3 for SVG object creation i.e. circle in scenario within in a HTML document.Output:
   Advantages: Before Mouse move over:Disadvantages:

1. After Mouse move over:

Moreover, animations, transitions, attributes can be added and manipulated with less efforts using a D3 framework. All the work of action handling can be done using helper functions. In above example, select() function performs the task of retrieval of an argument whereas append() adds attribute as a child to the selected argument. D3 focuses on abstraction and thus most of the inner actions or executions are hidden from the end user, thus making it more easy to use. The task of binding event is done in the above case with the help of .on() function which passes mouse events as an argument. Noticeably, anonymous function concepts are used in the D3 framework. Herein, anonymous function is passed as an argument.
More complex actions can be done using the D3 framework such as retrieval of data from a different format of dataset such as .csv or JSON file.

• D3 supports web standards such as HTML, CSS, SVG which are known to all programmers, thus it can be used easily by anyone. In short, D3 exposes the capabilities of web standards such as HTML5, CSS3 and SVG.
• It is quite lightweight and flexible with the code which is reusable, thus preferable.
• It gives a wider control to the user to manage the visualization and data then the other API’s or frameworks available.
• Being an open source framework, one can easily manipulate the source code of D3 as per his/her need.
• D3 is not compatible with older versions of browsers. In case, if someone wishes to visualize the data with backward compatibility, the visualization might necessarily be static, because of poor compatibility.
• Security is still a challenge for D3. Data can’t easily be hidden or protected using D3.

Applications: Its advantages is preferable in various data visualization fields. Some of the major domains wherein D3 is used is as follows:

• Basic charting and graph analytic visualizations.
• Network visualizations.
• Data dashboard development modules.
• Web Maps creation and synthesis.
• Interactive data representation.

Originally published at https://www.geeksforgeeks.org on January 14, 2019.

Pseudo code is a term which is often used in programming and algorithm based fields. It is a methodology that allows the programmer to represent the implementation of an algorithm. Simply, we can say that it’s the cooked up representation of an algorithm. Often at times, algorithms are represented with the help of pseudo codes as they can be interpreted by programmers no matter what their programming background or knowledge is. Pseudo code, as the name suggests, is a false code or a representation of code which can be understood by even a layman with some school level programming knowledge.
Algorithm: It’s an organized logical sequence of the actions or the approach towards a particular problem. A programmer implements an algorithm to solve a problem. Algorithms are expressed using natural verbal but somewhat technical annotations.
Pseudo code: It’s simply an implementation of an algorithm in the form of annotations and informative text written in plain English. It has no syntax like any of the programming language and thus can’t be compiled or interpreted by the computer.

Advantages of Pseudocode

• Improves the readability of any approach. It’s one of the best approaches to start implementation of an algorithm.
• Acts as a bridge between the program and the algorithm or flowchart. Also works as a rough documentation, so the program of one developer can be understood easily when a pseudo code is written out. In industries, the approach of documentation is essential. And that’s where a pseudo-code proves vital.
• The main goal of a pseudo code is to explain what exactly each line of a program should do, hence making the code construction phase easier for the programmer.

Disadvantages of Pseudocode

• Pseudocode does not provide a visual representation of the logic of programming.
• There are no proper format for writing the for pseudocode.
• In Pseudocode their is extra need of maintain documentation.
• In Pseudocode their is no proper standard very company follow their own standard for writing the pseudocode.

How to write a Pseudo-code?

1. Arrange the sequence of tasks and write the pseudocode accordingly.
2. Start with the statement of a pseudo code which establishes the main goal or the aim.

Example:

This program will allow the user to check the number whether it's even or odd.

Example:

if "1"
    print response
        "I am case 1"

if "2"
    print response
        "I am case 2"

1. Use appropriate naming conventions. The human tendency follows the approach to follow what we see. If a programmer goes through a pseudo code, his approach will be the same as per it, so the naming must be simple and distinct.
2. Use appropriate sentence casings, such as CamelCase for methods, upper case for constants and lower case for variables.
3. Elaborate everything which is going to happen in the actual code. Don’t make the pseudo code abstract.
4. Use standard programming structures such as ‘if-then’, ‘for’, ‘while’, ‘cases’ the way we use it in programming.
5. Check whether all the sections of a pseudo code is complete, finite and clear to understand and comprehend.
6. Don’t write the pseudo code in a complete programmatic manner. It is necessary to be simple to understand even for a layman or client, hence don’t incorporate too many technical terms.

Originally published at https://www.geeksforgeeks.org on December 18, 2018.

Overview: Apache Hadoop is an open source framework intended to make interaction with easier, However, for those who are not acquainted with this technology, one question arises that what is big data ? Big data is a term given to the data sets which can’t be processed in an efficient manner with the help of traditional methodology such as RDBMS. Hadoop has made its place in the industries and companies that need to work on large data sets which are sensitive and needs efficient handling. Hadoop is a framework that enables processing of large data sets which reside in the form of clusters. Being a framework, Hadoop is made up of several modules that are supported by a large ecosystem of technologies.

Introduction: Hadoop Ecosystem is a platform or a suite which provides various services to solve the big data problems. It includes Apache projects and various commercial tools and solutions. There are four major elements of Hadoop i.e. . Most of the tools or solutions are used to supplement or support these major elements. All these tools work collectively to provide services such as absorption, analysis, storage and maintenance of data etc.

Following are the components that collectively form a Hadoop ecosystem:

• HDFS: Hadoop Distributed File System
• YARN: Yet Another Resource Negotiator
• MapReduce: Programming based Data Processing
• Spark: In-Memory data processing
• PIG, HIVE: Query based processing of data services
• HBase: NoSQL Database
• Mahout, Spark MLLib: Machine Learning algorithm libraries
• Solar, Lucene: Searching and Indexing
• Zookeeper: Managing cluster
• Oozie: Job Scheduling

Note: Apart from the above-mentioned components, there are many other components too that are part of the Hadoop ecosystem.

All these toolkits or components revolve around one term i.e. Data. That’s the beauty of Hadoop that it revolves around data and hence making its synthesis easier.

HDFS:

• HDFS is the primary or major component of Hadoop ecosystem and is responsible for storing large data sets of structured or unstructured data across various nodes and thereby maintaining the metadata in the form of log files.
• HDFS consists of two core components i.e.

1. Name node
2. Data Node

• Name Node is the prime node which contains metadata (data about data) requiring comparatively fewer resources than the data nodes that stores the actual data. These data nodes are commodity hardware in the distributed environment. Undoubtedly, making Hadoop cost effective.
• HDFS maintains all the coordination between the clusters and hardware, thus working at the heart of the system.

YARN:

• Yet Another Resource Negotiator, as the name implies, YARN is the one who helps to manage the resources across the clusters. In short, it performs scheduling and resource allocation for the Hadoop System.
• Consists of three major components i.e.

1. Resource Manager
2. Nodes Manager
3. Application Manager

• Resource manager has the privilege of allocating resources for the applications in a system whereas Node managers work on the allocation of resources such as CPU, memory, bandwidth per machine and later on acknowledges the resource manager. Application manager works as an interface between the resource manager and node manager and performs negotiations as per the requirement of the two.

MapReduce:

• By making the use of distributed and parallel algorithms, MapReduce makes it possible to carry over the processing’s logic and helps to write applications which transform big data sets into a manageable one.
• MapReduce makes the use of two functions i.e. Map() and Reduce() whose task is:

1. Map() performs sorting and filtering of data and thereby organizing them in the form of group. Map generates a key-value pair based result which is later on processed by the Reduce() method.
2. Reduce(), as the name suggests does the summarization by aggregating the mapped data. In simple, Reduce() takes the output generated by Map() as input and combines those tuples into smaller set of tuples.

PIG:

• Pig was basically developed by Yahoo which works on a pig Latin language, which is Query based language similar to SQL.
• It is a platform for structuring the data flow, processing and analyzing huge data sets.
• Pig does the work of executing commands and in the background, all the activities of MapReduce are taken care of. After the processing, pig stores the result in HDFS.
• Pig Latin language is specially designed for this framework which runs on Pig Runtime. Just the way Java runs on the JVM.
• Pig helps to achieve ease of programming and optimization and hence is a major segment of the Hadoop Ecosystem.

HIVE:

• With the help of SQL methodology and interface, HIVE performs reading and writing of large data sets. However, its query language is called as HQL (Hive Query Language).
• It is highly scalable as it allows real-time processing and batch processing both. Also, all the SQL datatypes are supported by Hive thus, making the query processing easier.
• Similar to the Query Processing frameworks, HIVE too comes with two components: JDBC Drivers and HIVE Command Line.
• JDBC, along with ODBC drivers work on establishing the data storage permissions and connection whereas HIVE Command line helps in the processing of queries.

Mahout:

• Mahout, allows Machine Learnability to a system or application. Machine Learning, as the name suggests helps the system to develop itself based on some patterns, user/environmental interaction or om the basis of algorithms.
• It provides various libraries or functionalities such as collaborative filtering, clustering, and classification which are nothing but concepts of Machine learning. It allows invoking algorithms as per our need with the help of its own libraries.

Apache Spark:

• It’s a platform that handles all the process consumptive tasks like batch processing, interactive or iterative real-time processing, graph conversions, and visualization, etc.
• It consumes in memory resources hence, thus being faster than the prior in terms of optimization.
• Spark is best suited for real-time data whereas Hadoop is best suited for structured data or batch processing, hence both are used in most of the companies interchangeably.

Apache HBase:

• It’s a NoSQL database which supports all kinds of data and thus capable of handling anything of Hadoop Database. It provides capabilities of Google’s BigTable, thus able to work on Big Data sets effectively.
• At times where we need to search or retrieve the occurrences of something small in a huge database, the request must be processed within a short quick span of time. At such times, HBase comes handy as it gives us a tolerant way of storing limited data.

Other Components: Apart from all of these, there are some other components too that carry out a huge task in order to make Hadoop capable of processing large datasets. They are as follows:

• Solr, Lucene: These are the two services that perform the task of searching and indexing with the help of some java libraries, especially Lucene is based on Java which allows spell check mechanism, as well. However, Lucene is driven by Solr.
• Zookeeper: There was a huge issue of management of coordination and synchronization among the resources or the components of Hadoop which resulted in inconsistency, often. Zookeeper overcame all the problems by performing synchronization, inter-component based communication, grouping, and maintenance.
• Oozie: Oozie simply performs the task of a scheduler, thus scheduling jobs and binding them together as a single unit. There is two kinds of jobs .i.e Oozie workflow and Oozie coordinator jobs. Oozie workflow is the jobs that need to be executed in a sequentially ordered manner whereas Oozie Coordinator jobs are those that are triggered when some data or external stimulus is given to it.

Originally published at https://www.geeksforgeeks.org on December 18, 2018.

This article will focus on Java memory management, how the heap works, reference types, garbage collection, and also related concepts.

Why Learn Java Memory Management?
We all know that Java itself manages the memory and needs no explicit intervention of the programmer. Garbage collector itself ensures that the unused space gets cleaned and memory can be freed when not needed. So what’s the role of programmer and why a programmer needs to learn about the Java Memory Management ? Being a programmer, you don’t need to bother with problems like destroying objects, all credits to the garbage collector. However the automatic garbage collection doesn’t guarantee everything. If we don’t know how the memory management works, often we will end up amidst things that are not managed by JVM (Java Virtual Machine). There are some objects that aren’t eligible for the automatic garbage collection.

Hence knowing the memory management is essential as it will benefit the programmer to write high performance based programs that will not crash, or if does so, the programmer will know how to debug or overcome the crashes.

Introduction:

In every programming language, the memory is a vital resource and is also scarce in nature. Hence it’s essential that the memory is managed thoroughly without any leaks. Allocation and deallocation of memory is a critical task and requires a lot of care and consideration. However in Java, unlike other programming language, the JVM and to be specific Garbage Collector has the role of managing memory allocation so that the programmer needs not to. Whereas in other programming languages such as C the programmer has direct access to the memory who allocates memory in his code, thereby creating a lot of scope for leaks.

The major concepts in Java Memory Management :

• JVM Memory Structure
• Working of Garbage Collector

Java Memory Structure:

JVM defines various run time data area which are used during execution of a program. Some of the areas are created by the JVM whereas some are created by the threads that are used in a program. However, the memory area created by JVM is destroyed only when the JVM exits. The data areas of thread are created during instantiation and destroyed when the thread exits.

Let’s study these parts of memory area in detail:

• It is a shared runtime data area and stores the actual object in a memory. It is instantiated during the virtual machine startup.
• This memory is allocated for all class instances and array. Heap can be of fixed or dynamic size depending upon the system’s configuration.
• JVM provides the user control to initialize or vary the size of heap as per the requirement. When a new keyword is used, object is assigned a space in heap, but the reference of the same exists onto the stack.
• There exists one and only one heap for a running JVM process.

Heap :

Scanner sc = new Scanner(System.in);

The above statement creates the object of Scanner class which gets allocated to heap whereas the reference ‘sc’ gets pushed to the stack.

Note: Garbage collection in heap area is mandatory

• It is a logical part of the heap area and is created on virtual machine startup.
• This memory is allocated for class structures, method data and constructor field data, and also for interfaces or special method used in class. Heap can be of fixed or dynamic size depending upon the system’s configuration.
• Can be of a fixed size or expanded as required by the computation. Needs not to be contiguous.

Method Area:

Note: Though method area is logically a part of heap, it may or may not be garbage collected even if garbage collection is compulsory in heap area.

• A stack is created at the same time when a thread is created and is used to store data and partial results which will be needed while returning value for method and performing dynamic linking.
• Stacks can either be of fixed or dynamic size. The size of a stack can be chosen independently when it is created.
• The memory for stack needs not to be contiguous.

JVM Stacks:

Native method Stacks:

Also called as C stacks, native method stacks are not written in Java language. This memory is allocated for each thread when its created. And it can be of fixed or dynamic nature.

Program counter (PC) registers:

Each JVM thread which carries out the task of a specific method has a program counter register associated with it. The non native method has a PC which stores the address of the available JVM instruction whereas in a native method, the value of program counter is undefined. PC register is capable of storing the return address or a native pointer on some specific platform.

• JVM triggers this process and as per the JVM garbage collection process is done or else withheld. It reduces the burden of programmer by automatically performing the allocation or deallocation of memory.
• Garbage collection process causes the rest of the processes or threads to be paused and thus is costly in nature. This problem is unacceptable for the client but can be eliminated by applying several garbage collector based algorithms. This process of applying algorithm is often termed as Garbage Collector tuning and is important for improving the performance of a program.
• Another solution is the generational garbage collectors that adds an age field to the objects that are assigned a memory. As more and more objects are created, the list of garbage grows thereby increasing the garbage collection time. On the basis of how many clock cycles the objects have survived, objects are grouped and are allocated an ‘age’ accordingly. This way the garbage collection work gets distributed.
• In the current scenario, all garbage collectors are generational, and hence, optimal.

Working of a Garbage Collector:

Note: System.gc() and Runtime.gc() are the methods which requests for Garbage collection to JVM explicitly but it doesn’t ensures garbage collection as the final decision of garbage collection is of JVM only.

Knowing how the program and it’s data is stored or organized is essential as it helps when the programmer intends to write an optimized code in terms of resources and it’s consumption. Also it helps in finding the memory leaks or inconsistency, and helps in debugging memory related errors. However, the memory management concept is extremely vast and therefore one must put his best to study it as much as possible to improve the knowledge of the same.

Originally published at https://www.geeksforgeeks.org on December 13, 2018.

“There are innumerable novels, books and magazines which are just kept in the rack/shelves of bookstores which hardly ever get noticed by anyone !”, this statement was put forth by Anushka Sharma, who was casted as a writer in a biopic movie named ‘Sanju’.

Isn’t it true? It makes sense. But how and why all that is happening? Marketing and advertisements, is the sole answer. Even a best story needs a great advertisement or publisher media to reach its audience else might end up amidst dust. And many of ‘em fail to reach the audience due to poor advertisements.

We all are living in an era which is filled with densely populated nations and the global market running it. Every marketer or enterprise focuses to utilize it’s assets to make their product best. But in this competitive era, where every next second the stock market faces ups and downs, maintaining the profit quo isn’t easy. Because for creating and maintaining every product, there is a dire need of money i.e investment. And money and investments aren’t that easy to be achieved in this competitive globe. So, how this requirement is fulfilled? The major solution to the above scenario is ‘marketing’.

Marketing, is the strategy which focuses on driving the product towards its destined customers.

However marketing isn’t a term which can be elaborated in one sentence but is a major field of scope. Every company or enterprise has a different section for its marketing, because of the usefulness of the same. A product which is made well is of no use unless and until it reaches its target customers. Marketing, hence is essential. One of the most important factor or key concept of marketing is, Advertising.

Wikipedia suggests advertising as, “a marketing communication that employs an openly sponsored, non-personal message to promote or sell a product, service or idea.” In simple, if we consider marketing as a vehicle directing or driving an enterprise or product towards a destination, then advertisement is the fuel running that vehicle.

In order for a product to reach towards its customers, advertisement is vital. But though the meaning of advertisement is static, but its methodology has changed in the modern times. In the good old days, people used to live in much smaller and well-communicated communities. Almost all people knew each other in the places that they lived in. For a salesman who wanted to advertise a product, the means of communication would be based on social grounds. Persuasion was evident and it was not entirely of an objective nature. However, it was a rational and sensible one. Things or methodology of advertisement used to be more genuine and down to Earth. The salesman would maintain ties with the customer and face to face interaction was more evident. Lying to someone with respect to a product is easy in the modern world where online methodology is used but its hard to frame a product with lies face to face. In the older era, advertisement’s influence was rational and genuine. A product was sold with respect to its genuine facts instead of exaggerated ones.

Modern advertising is of a devilish nature.

And most of us fall prey to it. It simply kills our ability to think critically. Modern advertising too often addresses our emotions, not intellect. Different ways are used to make a customer fall into the trap of a product.

In a scenario conducted in the states of America, two pens were placed to sell. One was of a good quality whilst the other was of inferior build. But the inferior one was advertised with a vibrant colorful surrounding which resembled rich look and feel, whereas the good pen was sold without any embellishment. The embellishment resulted into the high sale of inferior product and about 70–80% people bought it though having knowledge of its inferior quality.

Simply, this experiment proved that embellishment lures us into a trap.

Other methodologies which influence us negatively is using ‘idols’ or ‘influencers’ of a society as a means of persuasion. Like sportsman Virat Kohli promoting a brand of ‘Manyavar’ or Christiano Ronaldo expressing in a fake candid that he loved every single bite of ‘KFC Chicken wings’. Another way of persuasion is using some living beings as ‘objects’. People would be willing to purchase anything as long as there is an attractive model ‘next to it.’ Another persuasion technique is the repetitive display of products everywhere. Yet we easily go for them.

Ads have maybe crossed their limit and have become an annoyance. In a fast paced life, no one wants to have ad to slow them down. Suppose there’s a student looking for tutorial videos on Youtube while preparing for exam and boom! An advertisement video of 15 seconds pops up. There’s no measure to such annoyances. Another scenario where there’s a 10 year old kid watching Television and an ad of deodorant comes up which shows up no facts of the deodorant’s ingredients but is an attempt to establish among viewers a feeling of that deodorant making an impact on girls. All these ads create a negative impact on the young minds.

Food, clothing and shelter were the basic need of society, but funny how embellishments have become necessities in our modern world.

Fair & Lovely which never makes anyone fair somehow establishes a feeling that the amount of melanin pigment in someone’s skin ie. complexion is a measure to success. Step Up, Stay on, etc. are the products which makes nothing but fake promises of making someone taller or slimmer and so on. Due to exaggerated ads, its difficult for customers to determine which product is good and which’s not. Thanks to standards and certifications like ISO 2008:9001, 6 Sigma and so on, products’ efficiency and reliability can be measured. Even a sales pitch for investment is full of lies too, at times. If ads haven’t been making fake promises, no writer would’ve ever bothered to write a quote on it stating

‘Advertising – A judicious mixture of flattery and threats’.

-Northrop Fyre

We surely need to reassess the standards on which we rely when buying what we “need.” In the meantime, and until we reassess our standards, let us keep our hands on our pockets in an attempt to protect our money that might be wasted on something we don’t even need.

Ps: I am just a student and no such marketing expert, its just an attempt to put down my observations and ideas. There are many pros of advertisements too, will pen them down sooner.

Thanks for reading. :)

Influence of advertising on our thinking was originally published in The Virtual Ink on Medium, where people are continuing the conversation by highlighting and responding to this story.

“Do you know, I saw a man in his mid-40s in the same garden where we used to go the most for spending our quality time. He seemed from a poor background as was dressed in ripped-torn shirt-jeans”, Aarti kept muttering as I gulped down the last sip of coffee. I kept wondering if her voice is more stimulating than the coffee.

And after the cup of coffee was empty, I maintained my silence, but was eager to listen her more and more. She seemed disturbed today, or I could say, she seemed more pretty in her disturbed mood too. I didn’t wanted her to get alright so easily, as her disturbed looks were taking every bit of my attention. Now I was wondering how can a disturbed or frowned face be so pretty.

Too many questions that I keep asking myself when being with her, darn it !

“Will you say something? What is it ? A monologue platform for me or what?”, she frowned more this time. Or faked a frown. I chuckled terribly hard after looking at her so called ‘frowned expressions’.

“Okay madam. Here’s all ears to you ! Why are you so upset, what happened?”, I asked with curiosity, expecting some more of her furiosity.

“Let me speak first, don’t interrupt !”, she ordered. “So that man said me a no ! I mean, what the heck ! How the hell can he reject my request ?!”, she grumbled. I never experienced her in such a grumpy mood ever.

“What was your request?”, I asked.

“Don’t interrupt, boy !”, she fired back.

“Sssshhhhhhh !”, I teased.

“Haha, funny! So the thing is, that man was drawing the outline of a teenage girl sitting there with her dad without her knowledge. It was such a mesmerising sketch, I swear. It was like a candid drawn in few minutes. He later offered that sketch to her and she was extremely moved by his artistic work.”

“How much money he took for the sketch?”, I asked.

“Shut up ! He did it for his passion. Took not a single penny. Denied any offer by that girl and her dad. I too wanted to have my sketch drawn by someone. So I went up to him and asked for a sketch. He didn’t made for me. He went away instantly.”, She sighed.

“Maybe he is doing it for his passion solely.”, I asserted.

“Indeed! He does it for his passion and chooses for whom should he draw by himself, not on anyone’s request. I always wanted to see a sketch of mine. Today I felt I’ll be lucky to have one. But no !”, she cried.

I wanted to tell her that sketching was in my veins, at times. It was a secret which I hid all the time from her, from my family , from everyone. I made only one sketch till date and after that I never dared to make another one.

Sketching is an aesthetic art of drawing the minutest details with every string of your attention. Every line, shade and emphasis not only makes an impact on the sketch but also on your heart. That’s the reason why I never dared to make any sketch of any person again. Julie, my first love of my life, I drew it for once. I drew it in curiosity. Didn’t knew my sketch will be so impactful that she’ll be in tears after seeing it and I’ll be in tears seeing her.

Julie and I were in a tragic relationship after that and thus, when it came to an end, forgetting her became excessively hard as all the shades, outlines and lines were still fresh in my memory. That love for her , and the sketch, and her place in my heart, lasted for a longtime.

I wished to draw a sketch for her, as it was one of her wish that she professed up to me. I don’t want any wish of her to be incomplete, but the ripe pain of Julie always made me not to draw. So I still avoided it.

Days passed. And our love became stronger. Aarti and I went to that garden many a times, and she whenever saw that man sitting there drawing for someone, felt pity on herself. She always hoped that she’ll be one of his sketch, but all in vain.

It became her expectation that her sketch will be drawn by someone, and it was broken by that man unknowingly. She started questioning herself and me, whether she’s less than any of those people whose sketch he draws. And thus, it was my need to boost her up. Though I was firm not to make anything for her, I still decided to pick up the brush and pencil for the second time.

I decided that I’ll make it for her and gift it in the next month, as her birthday gift. Because it was irresistibly hard for me to see her questioning herself .

Only seven days were remaining for her birthday so I was all excited to make up the sketch.

With her eyes closed, her golden hair spread on the pillow, half-turned towards me. She must be exhausted so that she slept early that day, as it’s only evening. The candlelight spilled shadows near her eyelashes, that made her brows and cheeks glow.

Thats what the candid picture of her saved in my gallery depicted.

I turned and went to my trunk, then knelt to open it. Inside, I found my sketchbook and pencil case. Then I picked up a straight-backed chair and set it down next to the bed. And began to put on paper what I couldn’t say in words.

That’s why I love sketching, because words don’t express every detail which the sketch can.

In about 10–12 hours of continuous sketching, I portrayed all what was clear in my memory and picture. And in that manner, I finished sketching her. Still something felt incomplete.

No my sketch wasn’t incomplete, but her expressions in real life were. Her innocent smile was missing since a long while. I badly needed her smile, because that became the reason I moved on.

I gifted her the sketch on her birthday. And yes, like what happened with Julie, Aarti too burst out in tears for a long span which made my eyes teary as well. But yes, after gifting her the sketch, something felt complete.

Yes, her smile felt complete. And my world, seemed complete. “Thanks for everything”, she said and hugged me for real long.

The next day we went to the same old garden and were reading aloud the happy birthday wishes and were watching pictures of gifts which she received. And every wish made her more and more happy or fulfilled. We were so lost in reading the wishes that we lost the track of time. The stars started twinkling and it was the time for us to bid goodbye to each other. We hugged each other before going to our respective home.

“Heyy”, she said.

“What?”

“See, there’s something left beside where we sat.”, she asserted and I looked back at the verandah where we sat.

We both went to see the thing which we didn’t noticed for a hour or two, maybe. It was a white sheet folded with a rubber band.

“That sketchman must’ve forgotten a sketch, we must return it yesterday ”, she reckoned.

I opened the rubber band. And and !

We both froze in amazement. It was a sketch of her drawn by that man. It was really an amazing masterpiece drawn by him.

Alongwith it, a note was attached.

“Your expressions showed greed when you asked for sketch, thus I denied your request. Rest of the days you lacked a smile, so I couldn’t find you worth my time. I draw the sketch of those who are complete, free from negativity and greed, and you seemed free from all worries today. And your smile expressed your beautiful soul. So here’s a little from my side. Hope you like it!”, the note read.

“I don’t sketch faces but the expressions”, reads the other note.

A smiley and his signature was on the third note.

Now, everything felt complete as a whole.

Greed and negativity are the biggest rivals of us all. The more we have greed, the more we expect. In order to live a happy life, greed and negativity needs to be eliminated.

And we must not search of happiness elsewhere, because the source of it resides within us. That’s what I tried to express with this bittersweet love story.

Thanks for reading y’all. Hope you liked it. Have a good day ahead readers.

Sketched. was originally published in The Virtual Ink on Medium, where people are continuing the conversation by highlighting and responding to this story.