Computer vision and object detection are increasingly used in the automation of business processes. Along with the dynamic development of technology, especially artificial intelligence, there are many new innovative business applications for this type of algorithms. In one of our recent posts, we outlined how to build a custom R-CNN based detector. In this article, we will walk you through how to do the same with YOLO v5, which has grown in strength in recent years. One of the important aspects over and above other solutions is the speed of inference. As part of this article, we will present the specified use case and all steps of its implementation. Enjoy your reading!

Use case

Signatures are still one of the most common methods of document authentication. Especially for enterprise applications, identification of signed and unsigned copies of documents in digital repositories may be time-consuming and a challenging task. However, automation supported by machine learning can make it easier. In this context, we describe the capabilities of YOLO v5 detector, and we will discuss issues of detecting invoice signatures. Let’s start with an example as shown in the figure below. For the purpose of this study, we prepared a fake invoice, fake data of the seller, buyer or the product itself. Let’s investigate now, how we can focus on hand-written signatures (which is fake too :-)).

Collecting Data

One of the first steps is to collect a set of images to train your model. For our use case, we have prepared a small 10 sample training set of invoices filled with different data and augmented this collection by a dedicated Python script. To make things easier, we used a Roboflow, which is a useful tool for data tagging Here you can upload your dataset and make a quick annotation process like on this GIF.

Additionally, we can generate more output images with random values ​​of rotation, saturation, exposure, noise, blur and other types of transformations. Moreover, with the help of this tool, we can determine our training, validation and test data split – the default is 70%, 10%, 10%. This is an important thing to prevent overfitting our model, you can read more about it on this blog. Thanks to the described tool, we can export our dataset in the YOLO v5 Pytorch format and put it in the directory of our project. The figure below shows the selection and target tree, where one of the most important files is the data.yml that will be used for training.

In our case after the first step of augmentation by a Python script, we generate 69 images. Additionally, we use a noise option in Roboflow, which allows us, in the end, export 169 annotated examples. Our images were ultimately split as shown in the figure below. 

And here you can see some of the train examples… 

Model configuration and architecture

The next step in the whole process is to define the configuration and architecture of the YOLO model. We may build our own network structure, although in version 5 we are provided with one of the following models:

• YOLOv5s,
• YOLOv5m,
• YOLOv5l,
• YOLOv5x.

We may use any of them but we must remember to assign appropriate target file parameter value called nc – number of classes. In our use case, it’s 1. The models differ from each other with the number of parameters used, speed of frames per second (FPS), accuracy and others… On the figure below you can see how they cope with the same COCO dataset. Click on the image below to learn more.

Training

What’s important, the Ultralytics company provides us with YOLO v5 developed in PyTorch, a framework specialized in machine learning. Thanks to this, we may download their repository from GitHub and train our own detector. Nothing simpler, but what else do we need for everything to work fine? 

To run each of the scripts we must install the dependencies contained in the requirements.txt. If we are using pip, we can use the following command in the terminal. Please remember that you must be in the project directory. 

pip install -r requirements.txt

In the installation process, one thing may be a problem. Namely, the PyTorch library…

The different operating system, package in use, programming language etc. may require a specified command. You will find everything you need in this tutorial.

Let’s assume you have passed all the requirements and start our training!

One line of code is enough to run this process, but we need to make sure that we have prepared two important files – data.yaml and for example yolov5l.yaml. The first one you should have from the data collection stage and the second one can be found in the YOLO v5 repository, in the Models directory. 

That’s it! Let’s run the training by the following command: 

python train.py --data dataset/data.yaml --cfg models/yolov5l.yaml --weights ''

The above command is the simplest possible, additionally, you can define the following options or parameters: 

• img-size,
• batch-size,
• epochs, 
• name,
• no-save,
• cache…

For the purposes of this article, we conducted a relatively short training for images resized to 416×416, batch size 32 and 1500 epochs. When the process is complete, the result will be a weight file named best.pt.

Perhaps you may stop your learning process if your weights are optimal for you at the moment. We stopped our training after 650 epochs, which took about 15 hours in total. Please note that the PC used does not have a dedicated graphics card.

Detailed metrics of our training:

• precision – 0.93507,
• recall – 0.96429,
• mAP_0.5 – 0.94755,
• mAP_0.5:0.95 – 0.48702,
  

where:

precision – measures how accurate are your predictions,

recall – measures how good you find all the positives,

mAP_0.5 – mean average precision for IoU* = 0.5,

mAP_0.5:0.95 – mean average precision for IoU* from 0.5 to 0.95 with a step size of 0.005,

* IoU (Intersection over Union) – measures the overlap between 2 boundaries. Find out more about these metrics here.

Inference on test images

Now, when we have our trained model trained after the learning stage, we may go to inference on test images. As you remember, one of the directories exported from Roboflow was called test. We can use it by putting in the terminal following line: 

python detect.py --weights best.pt --source dataset/test --conf 0.6 --img-size 600

where conf is model confidence – higher required makes fewer predictions. 

Finally, we may see the result visualization. The approximate inference time on one test image 416×416 was ~ 0.3 s, while for 700×700 ~ 0.8s.

Summary

For the implementation of the described detector, we used only 10 images, extending the dataset by augmentation process. Thanks to the Roboflow tool, it was possible to quickly annotate and export the data to the YOLO format. The learning process allowed us to obtain a target detector that effectively recognizes signatures on invoices.

Please note that with such a small dataset it is only adapted to recognize similar invoices. If we wanted to expand the possibilities of our detector, we would have to equip ourselves with a better graphics card and more, and more data. Additionally, you may be tempted to add a new class, e.g. to separate the objects into the Signature of the Seller and the Buyer.

Maybe you will see a new article in the future with more recognition classes. Stay tuned!

In Inero Software – software development agency, we are experts of digital transformation and smart automation. We provide knowledge and expertise on how to successfully use cutting edge technologies and data to shape corporate digital products of the future.

For more information, visit us on our website or follow us on LinkedIn. 

Artykuł Digital Cloud Document Repositories – how to identify signatures in scanned PDF documents pochodzi z serwisu Inero Software - Software Consulting.

The power of prediction by the numbers

Between 2005 and 2010 the computerization of processes began for good, turning many practices and tasks in the professional world from their analogue form to digital. Since then, one of the common aims of enterprises operating in the digital transformation era has often been phrased as “becoming a data-driven company,” i.e. to rely on hard data, while taking business decisions, rather than on intuition and observations alone. Better information management capabilities often translate to adding volume and growth, reducing costs, improving performance and product innovation – to name a few.

Increasingly better data processing tools have been developed over the past decade. As Peter Sondergaard famously said, “Information is the oil of the 21st Century, and analytics is the combustion engine.” Moreover, the prices for storing data continuously fall. And since more and more information continues to flood businesses – coming from simple sensors that measure temperature, telemetry, more advanced devices that analyze the condition of equipment and locate it in buildings, to phones with GPS and e-mails – analysts and managers gain increasingly more opportunities to use it for the purpose of business development.

How can we help you?

Let’s see this through the perspective of one of our prospective clients, a company that rents construction equipment, such as excavators, cranes, and trucks. Its management wants to improve and automate the quoting process or at least to enable making pricing decisions semi-automatically for the team of twenty sales representatives.

In order to answer a simple question: “How much will it cost to rent a piece of equipment for a specified period of time?” a number of factors must be taken into account. In addition to technical data, such as: the timing, location, travel duration and mileage, as well as load, combustion, etc., there are various other elements, such as insurance conditions or proceedings in case of damage, making it a fairly tricky calculation. In cases of increased risk – as with companies, which habitually return the equipment damaged – the managers may have to offer a higher price. On the other hand, the clients who always return equipment in pristine condition and pay on time should enjoy more favorable pricing. Moreover, some offers may seem beneficial, but the company might not profit from them due to one or two hidden factors.

With several dozens of such quote requests per day, and each bid being affected by a dozen or so factors, it’s ineffective for each salesman to prepare such a quote based solely on his/her calculations, past experience and intuition. In other words, the quoting process is too complex to be efficiently interpreted with human mental capabilities solely. It is, however, cut out for an approach known as data-driven (or data-directed) decision making – DDDM.

Base your actions upon mathematical reason thanks to data

Data-driven decision making in its core means that the basis for decisions should be researched and concluded from key data sets that show their projected value and how they might perform. Thanks to machine learning (ML) methods, which are used to collect and process data, we can not only verify which data elements really affect whether an offer is beneficial to our bottom line or not, but also validate our decisions before making them, avoid bias by making decisions based on huge amounts of current, real-time data; and diversify. You can dig deeper into the insights and establish additional sales opportunities, and identify underperforming areas that affect the overall sales of products. In addition to increasing efficiency, this approach can potentially teach us things that we’ve been misinterpreting for decades.

Data-driven decision making is being used in the fields of academia, business, and government to measure things in fine detail, as they occur. As a business technology, it has advanced exponentially in recent years, becoming ever more fundamental in various industries, including fields like medicine, transportation and equipment manufacturing.

State your goals, gather the proper data, structure the data

The key issue to remember while working with big data is that to extract genuine value from the data at your disposal, it must be relevant to your aims, which, in turn, should be defined prior to such analysis. If your data is incorrect, you’re going to be seeing a distorted view of reality.

Once the right questions are asked and business goals set, we approach the work with big data by structuring them. In order to ensure data quality, we categorize, organize and catalog data across different tables, removing or correcting data that is incomplete, or irrelevant. This is also an appropriate time to perform data targeting and adding more data elements to better describe phenomena, and find common patterns among the datasets. This is typically a moment when companies decide to use the services of an IT company, which can help in this process.

While preparing raw data for analysis, it’s important to remember, that various sets of data are interpreted differently (like information from underwater versus above water devices, etc.), moreover, different interpretation is required to process information from external sources or other IT systems that we want to integrate. The collection and structuring of data for the purposes of training Artificial Neural Network (ANN) – an ML model, which learns to perform tasks by considering examples – is already a big step that can illuminate certain things for us.

When data is prepared in such a way so that a neural network can learn, we use historical data, take into account the specificity of this data and the company’s operating model. This is an iterative process that we carry out many times to include all the necessary elements of the process, so that it brings the greatest value. Some solutions will appear only along the way.

Perform analytics-based sense making

Once we built accurate easily-transformed data sets, and measured it with statistical tools, we begin to analyze the information in order to answer the business questions identified earlier in the process. The insights – deep and intuitive understanding of phenomena – emerge not by mechanically applying analytical tools to data, but rather via an active process of engagement between data analysts and business managers. The uncovered knowledge can define the company’s development strategy, which generates value.

The creation of a proof of concept (PoC) with the use of innovative machine learning solutions requires competence in different areas, like data integration, understanding mathematics, SQL, and business processes. It requires many conversations and meetings, but above all – commitment from all parties involved. On the whole, this process takes many weeks; based on our experience – on average 3-6 months from the start to implementation. As with most investments, for several weeks there are no effects from the point of view of our customers, except for costs. Nonetheless, the concluding element of the process is the implementation of a final solution to the operational activity of the enterprise, or in other words – shedding light on the business questions, which made us embark on this quest in the first place.

The cost of building a PoC is roughly an equivalent to several dozen hours of developer’s work or to the cost of an advertising stand at international trade fairs (5,000-10,000 EUR). It’s not an exceptionally high cost compared to the benefits it can bring. At Inero Software we do not only design prototypes; but also test, improve and wrap the solutions with the graphical user interface (GUI), upon consulting it directly with end users. 

Calibrating to Industry 4.0

Markets and environments constantly change. It’s important to remember that for continued relevance in a changing landscape we can never be over-reliant on past experiences. And that even though future unfolds in front of our eyes, with the use of everyday analytics some of its aspects are within our grasp before they physically manifest.

Inero Software provides knowledge and expertise on how to successfully use cutting edge technologies and data to shape corporate digital products of the future.

In the blog post section you will find other articles about IT systems and more!

#DDDM, #DataDrivenDecisionMaking, #DataDrivenDecisionManagement, #MachineLearning, #ArtificialNeuralNetwork, #ANN, #ProofOfConcept, #MinimumViableProduct, #CustomSoftwareDevelopment

Artykuł Key steps to achieving data-driven decision making pochodzi z serwisu Inero Software - Software Consulting.

The interconnected nature of today’s economy continuously brings disruptions to traditional business models. For this reason it’s wise to keep an eye on tech development. After all, it’s technology that was named the main determinant of humanity’s future welfare.

In the previous entry we looked at the issue of digital transformation and its defining aspect – the use of computer modeling to enhance our business operations. The more and better data available, the more precisely we can model something, which in turn allows us to predict the way things will change or behave in future.

Over the past decade machine learning and artificial intelligence developments as well as real time access to IoT sensors have raised computer modeling to the next level, allowing us to model not only theoretically, but iteratively. By using the talents of data science on information flowing from the sensors, we can create what’s known as digital twins – real-time replicas of what exist in reality. 

A digital twin is a dynamic software model of physical assets, processes, people, places, systems, or devices, which uses physics data on how the components of the entity operate, respond to and interact with the environment. Data, bridging the real and digital world, is transferred flawlessly, allowing the virtual model to exist simultaneously with the physical entity. Thanks to the fast expanding body of data, such models can be continuously updated, allowing for more extended analyses.

Next Generation Computer Modeling

Digital twins’ usage varies from analyses and simulation of real world conditions, responding to changes, efficiency assessment, design, to maintenance, and many other applications, and there’s potential for literally billions of things to be represented by their digital twins. As a result of this, the use of digital twins is increasingly becoming a business imperative in many industries, as in case of the automotive industry, where they allow to design a product, simulate it and be able to test and validate its capabilities before the physical prototype of the product is built. All this to enable efficiency increases in manufacturing process and daily operations. At this point, the technology behind digital twins has expanded to include buildings, aircraft engines, container ships, and cities. One of the more ambitious plans with the use of this technology is The UK government’s plan to create a digital twin of its entire national infrastructure to prepare for future challenges.

Deliver-M8

One of the projects where we have implemented the digital twin concept is a delivery optimization system, designed to determine the most cost-effective routes, named Deliver-M8. It’s a software intended for fleet operations optimization in two structurally analogous sub-areas of the TSL sector: first and last mile delivery (FMD & LMD).

A demo version showing how the system works is available as a public site: deliverM8.com, where you can register and enter information on your organization’s logistics assets, such as: vehicle fleet, drivers and their working times, type of cargo, number of warehouses and their capacity, as well as customers’ availability, maximum time delay, and other variables, key for your operations. At your request the system will generate the most optimal delivery routes along with time schedules for your drivers within the time limits you set for it. Thanks to our API this information can be easily integrated with your other systems.

Say you are a dispatcher at a parcel warehouse with packages of different sizes and weights addressed to various locations in the city. You have a number of drivers working different shifts, and a number of vehicles at your disposal, plus a lot of places to visit. This and other input data is processed by the system, which offers the possibility of defining such constraints as the time window each recipient is available for delivery, the maximum delay time, and many other. Deliver-M8 assigns appropriate trucks to the drivers and plans the loading of packages in each car in the correct order. By clicking “Calculate plan” you can generate an optimal route for each driver.

Dynamic Balancing

Routes for individual trucks are divided on the map by color, you also see the drivers’ schedules broken down by hours. You can send the plan to each driver via email or a text message with a link. There’s a smartphone application that the driver can install as well, which asks them for their location, and shows them on the real-time web dashboard for dispatchers as a truck icon. The location is refreshed while driving, allowing you, the dispatcher, to see where the cars are at all times.

Our proprietary version of optimization algorithms implemented in Deliver-M8 allowed us to offer a very unique feature, i.e. dynamic balancing, which enables decisions to be made in real time in response to a logistics crisis. Upon clicking on the link they received from the dispatcher, the driver sees all their delivery addresses in a chronological order. At any point during the workday they are able to send feedback to the system using the application, noting visited addresses and unavailable customers, or postponement in the delivery. 

Deliver-M8 allows you to compare the feedback data with the original plan and detect if the registered disruptions threaten to change its outcome in real time. You can modify the plan at any point, including while the drivers are on the road. If you decide to change the original plan, the system generates a new one, taking into account real-time traffic information for which it queries external websites, as well as sends a new schedule to the drivers.

The above are examples of a broad range of Deliver-M8’s features, which go beyond the basics of route planning. Deliver-M8 is available for the FMD/LMD sector in a Software as a Service (SaaS) model, allowing for tests at an inexpensive cost and implementation either as a SaaS software with minor modifications, or as an in-house solution. To find out more about Deliver-M8 you can contact us at hi@inero-software.com

Advanced Developments Possible – Digital Twins

The digital twin concept is one of many new technologies changing the current business landscape. Applied to route optimization and paired with high-end machine learning algorithms it transfers into a powerful logistical tool, having potential to save your business time and money. As mentioned before, it is received with open arms far and wide – from the medical and insurance to security industries. As the concept is inseparably linked with IoT, endless influx of data will ensure its further deployment, and continuous advance of artificial intelligence and machine learning will safeguard more and more diverse appliances of digital twins in future.

Inero Software provides knowledge and expertise on how to successfully use cutting edge technologies and data to shape corporate digital products of the future.

In the blog post section you will find other articles about IT systems and more!

keywords: #digitaltransformation #IoT #data #datadrivendecisions #ProcessThinking #BusinessProcessModeling #BusinessProcessesOptimization #machinelearning #artificialintelligence #digitaltwins

Artykuł Digital Twins – Dynamic Models of Reality pochodzi z serwisu Inero Software - Software Consulting.