Hierarchical Planning for Heterogeneous Multi-Robot Routing Problems via Learned Subteam Performance

A recent paper by members of the DCIST alliance proposes a new hierarchical planner for task allocation problems where tasks correspond to heterogeneous multi-robot routing problems defined on different areas of a given environment. The researchers tackled this complex planning problem with a novel planner which breaks down the complexity of the original problem into […]

CORSAIR: Convolutional Object Retrieval and Symmetry-AIded Registration

This paper considers online object-level mapping using partial point-cloud observations obtained online in an unknown environment. We develop an approach for fully Convolutional Object Retrieval and Symmetry-AIded Registration (CORSAIR). Our model extends the Fully Convolutional Geometric Features (FCGF) model to learn a global object-shape embedding in addition to local point-wise features from the point-cloud observations. […]

Robust multimodal data association

A fundamental problem in robotic perception is matching identical objects or data, with applications such as loop closure detection, place recognition, object tracking, and map fusion. The problem becomes more challenging when matching is done jointly across multiple, multimodal sets of data, however, the robustness and accuracy of matching in the presence of noise and […]

ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description

A recent paper by members of the DCIST alliance develops a method for tightly coupled object shape and pose optimization. Inspired by DeepSDF, which uses neural networks to regress a Signed Distance Function (SDF) description of object shape, they propose a bi-level object shape model named ELLIPSDF, to support joint object pose and shape optimization. […]

Robust data association in high-outlier regimes

Establishing correspondence between two sets of data is a fundamental problem in robotics, and is required for fusing data across multiple DCIST agents to establish global situational awareness. Real-world data contains noise and outliers. The traditional linear assignment algorithms are not robust to high-outlier regimes, leading to incorrect correspondences. To address these issues,  members of […]

Active Bayesian Multi-class Mapping from Range and Semantic Segmentation Observation

Many robot applications call for autonomous exploration and mapping of unknown and unstructured environments. Information-based exploration techniques, such as Cauchy-Schwarz quadratic mutual information (CSQMI) and fast Shannon mutual information (FSMI), have successfully achieved active binary occupancy mapping with range measurements. However, as we envision robots performing complex tasks specified with semantically meaningful objects, it is […]

Multi-robot Scheduling for Environmental Monitoring as a Team Orienteering Problem

We develop an evolutionary algorithm for solving the multi-robot orienteering problem where a team of cooperative robots aims to maximize the total information collected by visiting a subset of given nodes within a fixed budget on travel costs. Multi-robot orienteering problems are relevant to applications such as logistic delivery services, precision agriculture, and environmental sampling […]

Optimizing Non-Markovian Information Gain Under Physics-Based Communication Constraints

A recent paper by members of DCIST proposes an exploration method that maintains communication between all robot team members and a static base station. By maintaining communication while exploring, robots are kept up to date on the progress of other team members and important information—e.g., survivors in a search and rescue mission—are quickly transmitted to […]

Active Exploration and Mapping via Iterative Covariance Regulation over Continuous SE(3) Trajectories

A recent paper by the members of the DCIST alliance develops a method for continuous-space optimal control of active information acquisition. They have developed “iterative Covariance Regulation (iCR)”, a novel method for an information-theoretic active perception performing multi-step forward-backward gradient descent. The problem is formalized as SE(3) trajectory optimization over a multi-step continuous control sequence […]

Heterogeneous robot teams for modeling and prediction of multiscale environmental processes

We present a framework to enable a team of heterogeneous mobile robots to model and sense a multiscale system. Their approach proposes a coupled strategy, where robots of one type collect high-fidelity measurements at a slow time scale and robots of another type collect low-fidelity measurements at a fast time scale, for the purpose of […]

Resilience in multi-robot multi-target tracking with unknown number of targets through reconfiguration

A recent paper by members of the DCIST alliance addresses the problem of maintaining resource availability in a networked multi-robot team performing distributed tracking of an unknown number of targets. Robots receive and process sensor measurements locally and exchange information to cooperatively track a set of targets using a distributed Probability Hypothesis Density (PHD) filter. […]

Distributed Certifiably Correct Pose-Graph Optimization

Recent work by members of the DCIST alliance presents the first certifiably correct algorithm for distributed pose-graph optimization (PGO), the backbone of modern collaborative simultaneous localization and mapping (CSLAM) and camera network localization (CNL) systems. The proposed method is based upon a sparse semidefinite relaxation that provably provides globally-optimal PGO solutions under moderate measurement noise […]

Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems

Recent work by members of the DCIST alliance presents Kimera-Multi, a multi-robot system that (i) is robust and capable of identifying and rejecting incorrect inter and intra-robot loop closures resulting from perceptual aliasing, (ii) is fully distributed and only relies on local (peer-to-peer) communication to achieve distributed localization and mapping, and (iii) builds a globally […]

Learning and Leveraging Environmental Features to Improve Robot Awareness

A recent paper by the members of the DCIST alliance studies how global dynamics and knowledge of high-level features can inform decision-making for robots in flow-like environments. Specifically, they investigate how coherent sets, an environmental feature found in these environments, inform robot awareness within these scenarios. The proposed approach is an online environmental feature generator […]

Coding for Distributed Multi-Agent Reinforcement Learning

A recent paper by the members of the DCIST alliance develops a multi-agent reinforcement learning (MARL) algorithm which uses coding theory to mitigate straggler effects in distributed training. Stragglers are delayed, non-responsive or compromised compute nodes, which occur commonly in distributed learning systems, due to communication bottlenecks and adversarial conditions. Coding techniques have been utilized […]

Intermittent Interactions on Multi-Agent Systems: Diffusion of Information and Consensus Control

Recent works by members of the DCIST alliance investigate methods to handle consensus and broadcast of information tasks in networks of mobile robots subject to intermittent communication. The effort of this work is to alleviate the restriction of an all-time connected network, letting agents interact periodically and taking into consideration the uncertainty associated with those […]

Intermittently Connected Mobile Robot Networks with Information Propagation Guarantees

DCIST researchers pioneered strategies for teams of mobile robots to form intermittently connected communication networks by leveraging their mobility.  Robots assigned to monitor and patrol large urban environments can leverage their movements to carry information to other robots that are not within their communication ranges. Our work shows intermittent connectivity between pairs of robots can […]

Learning Connectivity-Maximizing Network Configurations

A recent paper by members of the DCIST alliance develops a data-driven method for providing mobile wireless infrastructure on demand to multi-robot teams requiring communication in order to collaboratively achieve a common objective. While a considerable amount of research has been devoted to this problem, existing solutions do not scale in a manner suitable for […]

Dynamic Defender-Attacker Resource Allocation Game

A recent paper by members of the DCIST alliance proposes a new resource allocation game that studies a dynamic, adversarial resource allocation problem in environments modeled as graphs. By combining ideas from Colonel Blotto games with a population dynamics model, the proposed formulation incorporates: (i) dynamic reallocation in time-varying situations, and (ii) the presence of […]

Cooperative Systems Design in Adversarial Environments

The Colonel Blotto game describes a scenario where two opposing Colonels strategically allocate their limited resources across multiple battlefields. The game is compelling for a multitude of reasons, having numerous applications in military strategy. Optimal strategies in the Colonel Blotto game are highly complex – the game does not admit pure strategy equilibria in settings […]

Learning to swarm with knowledge-based neural ordinary differential equations

A recent paper by members of the DCIST alliance uses the deep learning method, knowledge-based neural ordinary differential equations (KNODE) to develop a data-driven approach for extracting single-robot controllers from the observations of a swarm’s trajectory. The goal is to reproduce global swarm behavior using the extracted controller. Different from the previous works on imitation […]

GNN based Coverage and Tracking Tracking in Heterogeneous Swarms

A recent paper by members of the DCIST alliance designs decentralized mechanisms for coverage control in heterogeneous multi-robot systems especially when considering limited sensing ranges of the robots and complex environments. These are part of the broader DCIST efforts for designing GNN-based control architectures which are, from the ground up, designed to operate in harsh […]

Learning Decentralized Controllers with Graph Neural Networks

A recent paper by members of the DCIST alliance develops a perception-action-communication loop framework using Vision-based Graph Aggregation and Inference (VGAI). This multi-agent decentralized learning-to-control framework maps raw visual observations to agent actions, aided by local communication among neighboring agents. The framework is implemented by a cascade of a convolutional and a graph neural network […]

Asynchronous and Parallel Distributed Pose Graph Optimization

A recent paper by members of the DCIST alliance has received a 2020 honorable mention from IEEE Robotics and Automation Letters. The paper presents Asynchronous Stochastic Parallel Pose Graph Optimization (ASAPP), the first asynchronous algorithm for distributed pose graph optimization (PGO) in multi-robot simultaneous localization and mapping. By enabling robots to optimize their local trajectory estimates […]

Non-Monotone Energy-Aware Information Gathering for Heterogeneous Robot Teams

A recent paper by members of the DCIST alliance considers the problem of planning trajectories for a team of sensor-equipped robots to reduce uncertainty about a dynamical process. Optimizing the trade-off between information gain and energy cost (e.g., control effort, energy expenditure, distance travelled) is desirable but leads to a non-monotone objective function in the […]

Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping

A recent paper by members of the DCIST alliance develops an open-source C++ library for real-time metric- semantic visual-inertial Simultaneous Localization And Mapping (SLAM). The library goes beyond existing visual and visual-inertial SLAM libraries (e.g., ORB-SLAM, VINSMono, OKVIS, ROVIO) by enabling mesh reconstruction and semantic labeling in 3D. Kimera is designed with modularity in mind […]

Asymptotically Optimal Planning for Non-myopic Multi-Robot Information Gathering

A recent paper by members of the DCIST alliance develops a novel highly scalable sampling-based planning algorithm for multi-robot active information acquisition tasks in complex environments. Active information gathering scenarios include target localization and tracking, active Simultaneous Localization and Mapping (SLAM), surveillance, environmental monitoring and others. The goal is to compute control policies for mobile robot […]

Active Exploration in Signed Distance Fields

When performing tasks in unknown environments it is useful for a team of robots to have a good map of the area to assist in efficient, collision-free planning and navigation. A recent paper by members of the DCIST alliance tackles the problem of autonomous mapping of unknown environments using information theoretic metrics and signed distance […]

Learning Multi-Agent Policies from Observations

A recent paper from the DCIST team introduces a framework for learning to perform multi-robot missions by observing an expert system executing the same mission. The expert system is a team of robots equipped with a library of controllers, each designed to solve a specific task. The expert system’s policy selects the controller necessary to […]

Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotors

A recent paper by members of the DCIST alliance develops the use of reinforcement learning techniques to train policies in simulation that transfer remarkably well to multiple different physical quadrotors. Quadrotor stabilizing controllers often require careful, model-specific tuning for safe operation. The policies developed are low-level, i.e., they map the rotorcrafts’ state directly to the […]

Planning with Uncertain Specifications (PUnS)

Consider the task of setting a dinner table. It involves placing the appropriate serving utensils and silverware according to the dishes being served. Some of the objects need to be placed in a particular order as they might be stacked on top of each other or due to cultural traditions. Many real-world tasks demonstrate such […]

Synthesis of a Time-Varying Communication Network by Robot Teams with Information Propagation Guarantees

A recent paper by Xi Yu and M. Ani Hsieh from the University of Pennsylvania presents a distributed control and coordination strategy that allows a swarm of mobile robots to form an intermittently connected communication network as they monitor a given environment. The approach assumes robots are tasked to patrol a set of perimeters in […]

Learning Decentralized Controllers with Graph Neural Networks

A recent paper by members of the DCIST alliance develops a method for distributed control of large networks of mobile robots with interacting dynamics and sparsely available communications. Their approach is to learn local controllers that require only local information and communications at test time by imitating the policy of centralized controllers using global information […]

Heterogeneity and Uncertainty in Perimeter Defense

Surveillance of perimeters and securing perimeters are important tasks in civilian and military defense applications, and it has become practical to deploy a large number of autonomous agents to address these problems using multi-robot systems.   A recent paper by members of the DCIST alliance formulates this scenario as a variant of multi-player pursuit-evasion games, where […]

Human Information Processing in Complex Networks

In this work, we study the structure of real-world communication systems to understand how information can be rapidly and efficiently communicated to humans, for example from swarms of drones or other agents. Humans constantly receive information from systems of interconnected stimuli or concepts — from language and music to literature and science — yet it […]

Resilient Active Information Acquisition with Mobile Robots

In the future, teams of heterogeneous robot teams will be operating in unknown and adversarial environments.   In failure prone or adversarial environments, the capability of resilience is crucial to ensuring the robots can complete their mission. Mission resilience to robot failures, sensor attacks or communication disruptions is currently an afterthought leading to optimal over-provisioned designs.   […]

Finite-Time Performance of Distributed Temporal Difference Learning with Linear Function Approximation

While many distributed reinforcement learning (RL) has emerged as one of the important paradigms in distributed control, we are only beginning to understand the fundamental behavior of these algorithms.  Two recent papers from the DCIST alliance provide important progress in this direction. In the multi-agent policy evaluation problem, a group of agents operate in a […]

Learning to Learn with Probabilistic Task Embeddings

To operate successfully in a complex and changing environment, learning agents must be able to acquire new skills quickly. Humans display remarkable skill in this area — we can learn to recognize a new object from one example, adapt to driving a different car in a matter of minutes, and add a new slang word […]

Localization and Mapping using Instance-specific Mesh Models

A recent paper by members of the DCIST alliance proposes an approach for building semantic maps, containing object poses and shapes, in real time, onboard an autonomous robot equippend with a monocular camera. Rich understanding of the geometry and context of a robot’s surroundigs is important for specification and safe, efficient execution of complex missions. This […]

Aerial Robot Prototype

Composable Autonomy in Heterogeneous Groups

In multi-robot Simultaneous Localization and Mapping (SLAM), a group of robots explore and map an unknown area. The group can benefit from its size by combining the robots’ maps to improve coverage and by each robot using shared information to improve its own localization. Most approaches to multi-robot SLAM consider homogeneous groups, in which all […]