A Cup Of Code

Part 2: Replication-Explained: Designing Distributed systems Overview: Original paper: The Google File System Authors: Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung > Practice agentic AI concepts hands-on for interviews and real-world development on AgenticPrep.io . A distributed file system(DFS) is a file system that allows you to access files stored in multiple hosts distributed across them. In contrast to other data stores like MySQL or NoSQL which can also be made distributed, the data is DFS is stored in raw form and as files instead of a table or a fixed-document format. The use cases where DFS is used are very vast. From storing media content like Youtube videos to Instagram images to blogs like these, anything that can be stored as a file, is big and is not suitable to store in other data stores, DFS is generally a preferred choice for everyone. Key points to consider while designing a DFS: Understanding from the perspective of using a distributed file system, the...

Objective: The objective of this work is to understand automated text anonymization system for protection of personal information of users and after anonymization, should still remain relevant in syntactic and semantics terms, without losing the conveyed meaning of text. Anonymized data can be used in many tasks such as data mining, machine learning, analysis, etc without revealing the identity of entities involved in the creation of data and be useful to improve the accuracy of the applied data analysis tool.   > Practice agentic AI concepts hands-on for interviews and real-world development on AgenticPrep.io . Approach: Beyond simply recognizing entities, variation in the types of documents (e.g., financial or medical) and type of identifying information poses a challenge for automated systems. One common approach is to develop domain specific anonymization tools, where one can utilize knowledge about the structure and information content of documents to construct h...

Motivation: Using Machine Learning will not require expert knowledge. All machine learning tasks follow the same basic flow. Difficult to find best fit hyper parameters. Hard to make hand made features. Fun. Automatic Machine Learning in progress: My motivation to write blog on this topic was Google's new project - AutoML . Google's AutoML project focuses on deep learning , a technique that involves passing data through layers of neural networks . Creating these layers is complicated, so Google’s idea was to create AI that could do it for them. There are many other open source projects, like AutoML and Auto-SKLEARN working towards a similar goal. Goal: The goal is to design the perfect machine learning “black box” capable of performing all model selection and hyper-parameter tuning without any human intervention.  AutoML draws on many disciplines of machine learning, prominently including Bayesian optimization - It is a sequential design strategy for gl...

Original Paper: An Optimization Model for Outlier Detection in Categorical Data Authors : Zengyou He, Xiaofei Xu, Shengchun Deng > Practice agentic AI concepts hands-on for interviews and real-world development on AgenticPrep.io . What are outliers? An out-lier is an observation that lies at an abnormal distance from other values in a random sample from a population. Before abnormal observations can be singled out, it is necessary to characterize normal observations. There are many ways to detect outliers in continuous variables but there exist only few techniques which can detect outliers in categorical variables.    Example : Suppose you have 1000 people choose between apples and oranges. If 999 choose oranges and only one person chooses apple, I would say that that person is an out-lier. We use measurement as a way to detect anomalies. With categorical data you have to explain why choosing an apple is considered an anomaly (that data point does not be...

> Practice agentic AI concepts hands-on for interviews and real-world development on AgenticPrep.io . Motivation. Database management systems are are systems used for accessing and manipulating information. The data can be manipulated using a set of keywords following of syntax rules. To perform operations on the database, it is required to learn the structured query language(SQL) . Hence, a user who do not know SQL cannot directly access information from the database. Natural Language Interface for Database(NLIDB). What is it? It is a proposed solution to the problem of accessing information in a database using natural language like English, having no technical knowledge about language like database languages like SQL. It is a tool which can understand a user's query in natural language, convert it into appropriate SQL query, so that the user can get the required information from the database. Various approaches for building NLIDB. Symbolic or Rule based approach. ...

Original Paper : A Modification of K Nearest Neighbor Classifier Authors : Hamid Parvin, Hoseinali Alizadeh, Befrouz Minati > Practice agentic AI concepts hands-on for interviews and real-world development on AgenticPrep.io . K - Nearest Neighbor Classifier KNN is one of the simplest classification algorithm that is used widely across the problems where there is little or no idea of prior distribution of the data. More about KNN can be learn from this or this . Here is a naive implementation of KNN from scratch. Take a look at it as it will improve your understanding about the internals of the algorithm. A cup of fact: You can tweet KNN's implementation from scratch. Here is one such implementation( not by me ) Modified K - Nearest Neighbor Classifier The main problem in KNN is that not all points or nearest neighbors have same importance. Also, when the dimensions of the data increases, the accuracy of KNN decreases ( curse of dimensionality). In this paper, ...

> Practice agentic AI concepts hands-on for interviews and real-world development on AgenticPrep.io . Python provides several keywords which enable a functional programming approach for python. These functions are all convenience features in that they can be written in Python fairly easily  and can be replaced by custom code with some more line of code. As most of the beginners to python are confused or do not efficiently use these features, I will here try to explain those features in a simple manner. LAMBDA : Lambda function also called anonymous function or unbounded function can be used as any normal function in python but without any name.Starting with the keyword lambda followed by parameters before the colon and return value after it. Syntax - lambda arguments : expression Ex: Getting any number modulo 3 ( of-course this can be done using simple %3 ) is equivalent to The above python expression can be stated as "declare a nameless function taking a param...