You can read more about our research topics and related publications on the following page.
Are major research results are available on our GitHub page.
Research Topics
CrowdMapping
The images captured by cameras mounted on windshields of road vehicles can be utilized in a crowdsourcing-like manner to create detailed 3D models that can describe frequently used parts of road networks and their surroundings. In a system that combines a large number of data contributors and modern image processing tools, the goal is to create a map database that supports the path planning and navigation of autonomous vehicles. This is what we are working on in the CrowdMapping project .
One of the basic requirements for the participation of a large number of data providers is that the network capacity required for data delivery should be minimal. Assuming image data collection, it is essential to pre-filter the data. We are developing hybrid image processing algorithms (using modern and classical tools) to solve this task.
The operation of algorithms that build and manage maps in the vehicles of individual data providers, i.e., cameras and embedded devices at the edge of the network, as well as on remote server computers and in the cloud, is also part of the research project. The edge-cloud structure thus formed raises several machine vision and infrastructure-related issues, which we address within the framework of the project.
The created map database can be kept up-to-date based on the data of the community participants. Under certain conditions, this solution allows for delays of even a tenth of a second and real-time data refresh. In the planning and navigation of autonomous vehicles, this solution results in safer, more accurate, and faster algorithm operation. In hybrid transportation, this phenomenon was examined along the lines of changing weather conditionsand opportunities for optimizing the route planning of interconnected and autonomous vehicles.
Robust and Efficient Vision Solutions
Some modern 3D modeling environments are prepared for differentiating the rendering of modeled objects based on certain parameters. In an environment containing objects and a camera determining the rendering viewpoint, such a parameter could be the position of the camera. Minimizing object occlusion can thus be seen as a parameter optimization task, for which machine learning-based solutions provide excellent results. Our system, which implements the environment and trained networks, is available on the OcclusionEnv GitHub page.
When it comes to visual systems used in autonomous vehicles, it is particularly critical to prepare the system for various visual impairments, such as fog, snow, or rain. However, collecting such examples in real life can be difficult and costly, so creating simulated images with such disturbances may be a practical solution. In the context of the research, we are developing novel solutions for simulating fog on images and examining different methods for automatically removing fog from images.
Object detection and segmentation are fundamental tasks in computer vision, for which there exist numerous state-of-the-art neural network-based methods. Nowadays, there is an increasing emphasis on how fast these methods can run on various low-power embedded devices, such as mobile devices and robots. In our research , we developed the first end-to-end neural network-based vision systems that were able to run in real-time on RoboCup Standard Platform League robots.
One limitation to the applicability of neural networks is the limited resources of the computing environment. To address this problem, we are researching techniques that can automatically reduce the number of unnecessary parameters and thus the size of the network, without compromising its efficiency. As a result of our research , we have developed a reinforcement learning-based network pruning solution that achieves excellent results.
Autonomous Vehicles
One significant requirement of modern image processing solutions, and in particular, computer vision, is the existence of large quantities of diverse data. A widely used approach to meet this need is to create simulated 3D urban maps and render realistic traffic situations and environments. In our group, we focus on examining the usability of such autonomous vehicle simulator environments, as well as synthesizing training databases.
One challenge with algorithms that perform well in simulation environments is that their effectiveness in real-life scenarios is not guaranteed. This is because the data generated by simulators is not realistic enough. Some Sim2Real (simulation-to-reality) techniques aim to address this problem. One significant research area for our team is finding solutions to transfer synthesized data to a realistic domain using our developed Label-Consistent Swapping Autoencoder architecture.
In the field of autonomous vehicles, the precise detection of objects in the environment is a critical task, which is particularly difficult due to partial or complete occlusions, which can cause important objects (such as pedestrians) to be lost, leading to accidents. However, it is possible for multiple autonomous vehicles to share their vision results with each other, thereby examining a given scene from multiple viewpoints. In our new research, we are investigating whether the novel NeRF (Neural Radiance Fields) based solutions can provide an effective solution to this problem.
Our group, as a participant in the international AI competition RoboCup, aims to research the feasibility of perception-action algorithms that leverage modern computer vision tools to address problems encountered in tasks involving the collaboration of multiple autonomous agents. In our research, we develop procedures that can perform efficient object detection , optimally utilizing the limited computational capacity available in the embedded environment of the agents. Actions based on these perceptions are capable of improving the cooperative and competitive performance of the robots in various tasks.
Medical Vision
Segmentation of the retina's blood vessels is one of the most important steps in the early diagnosis of eye-related diseases. Numerous machine learning-based image processing techniques can be applied for this type of segmentation. The investigation and further development of these techniques is one of the important research goals of our team. Novel network structures such as SA-UNet and CAR-UNet, in which our team participated in the development, can be used for this purpose.
Assisting medical decision-making with automatic methods is a research area with immense societal benefit. Therefore, in our laboratory, we have been working on several vision systems capable of automatically detecting malignant skin lesions, thereby supporting the work of dermatologists.
In animal husbandry, it is important to measure and record the physical parameters of breeding animals, which can be costly and in certain cases, pose a safety risk. In a joint project with the Faculty of Veterinary Science, we are developing software that can automatically estimate important anatomical landmarks of certain animals using side and top view camera images, thereby facilitating efficient work for veterinarians and breeders.
Smart City and Geographic Informatics
Smart city applications are extremely important for our research group. In this light, we participated in the Smart City - Smart Administration project, which was realized in collaboration with BME and the National University of Public Service. As part of this project, we also created several educational materials that are publicly available.
One of the important missions of our research group is to investigate the sustainability-critical applications of various artificial intelligence methods. Therefore, we have established active cooperation with the Budapest Transport Center (BKK) and the Department of Environmental Economics at the Budapest University of Technology and Economics (BME) to study the effects of possible developments in the Budapest BuBi network using neural network-based predictions.
One of the most important directions of development related to smart cities involves technologies enabling modern traffic control systems. In connection with this, automated identification of road vehicles is a task of paramount importance. There are numerous solutions available for recognizing license plates based on image processing, and we are working on developing these solutions by exploiting new possibilities.
Modern Localization and Mapping in Autonomous Driving
Our team has developed a new framework for simultaneous localization and mapping (SLAM) task, which efficiently supports image processing procedures using machine learning thanks to its highly modular composition. During the development of the ATDN vSLAM framework, we are looking for solutions to many modern image processing problems.
Visual odometry is the first and one of the most important steps in vSLAM procedures. In the ATDN framework, we apply solutions for determining the position and orientation of agents that are based on new algorithms containing deep neural networks.
The mapping subsystem's task is to create a compact representation of the areas already traversed by the data-gathering agent. To solve this task, we are developing a machine learning-supported algorithm that creates a neural-based, general map using popular learning methods.
