accumulate

blurry reconstruction

illustrate

2D convolutions are not designedto directly leverage the tiled structure of this feature space.Instead, we propose to first learn to expand this structured

The result-ing tensor is at a reduced resolution, but in contrast to strid-ing or pooling

folding the spatialdimensions of convolutional feature maps into extra featurechannels

information loss is not a necessarycondition to learn representations capable of generalizing todifferent scenarios

Lv between the magnitudeof the pose-translation component of the pose network pre-diction ˆtand the measured instantaneous velocity scalar vmultiplied by the time difference between target and sourceframes ∆Tt→s,

instantaneous

Mt is a binary mask that avoids computing the pho-tometric loss on the pixels that do not have a valid map-ping

an appearance matching loss term Lp

Synthesis

Photometric

impractical

alleviating

velocity

instantaneous

and rely on the ground-truth LiDARmeasu,rements to scale their depth estimates appropriatelyfor evaluation purpose

ad,ditional

where a depth and pose network are simultaneouslylearned from unlabeled monocular videos

vided an alternative strategy involving training a monocu-lar depth network with stereo cameras, without requiringground-truth depth labels

theavailability of target depth labels became challenging

amortize

not require supervised pretraining on ImageNetto achieve state-of-the-art results

DDAD enables much more accuratedepth evaluation at range

Ourthird contribution is a new dataset:

is a novelloss

inherent scale ambiguity

velocity

propagate

PackNet, for high-resolution self-supervisedmonocular depth estimation

estimation have mostly focused on engineering the lossfunction

Although self-supervised, ourmethod outperforms other self, semi, and fully supervisedmethods on the KITTI benchmark

prerequisite

DDAD

inductive

symmetrical

ubiquitous

ex-ploit redundancies

catastrophic

exploit the motion parallax

which is a degenerate case for traditional structure from motion

identical

concatenation

Higher resolution gives the Base-*methods an advantage in depth accuracy, but on the otherhand these methods are more prone to outliers.

This enhances the smooth-ness of estimated flow fields and the sharpness of motiondiscontinuities

Scale invariant gradient loss

discontinuities

Therotation r = θv is an angle axis representation with angle θand axis v. The translation t is given in Cartesian coordi-nates.

is identical to the boot-strap net, but it takes additional inputs.

By feeding optical flow estimate into the secondencoder-decoder we let it make use of motion parallax

gets the imagepair as input and outputs the initial depth and motion es-timates.

takes as input the optical flow, its confidence, the im-age pair, and the second image warped with the estimatedflow field. Based on these inputs it estimates depth, sur-face normals, and camera motion.

Schematic representation

The network estimates not only depthand motion, but additionally surface normals, optical flowbetween the images and confidence of the matching

succes-sive

egomotion

integrated

prone

homogeneous

scale-invarian

emphasizes

invariant

magnitude

deviations

penalize

and stimulate synergy of thetwo tasks without over-fitting to a specific scenario

images with unknown camera motion can be determinedonly up to scale

Parameterization

discontinuities

ambiguity

The second encoder-decoder

recursively

DeMoN takes an image pair as input and predicts the depth map of the first image and the relativepose of the second camera.

explicitly

implicitly

temporal

semi-dense

Outliers

unconstrained pair of images

In this paper, we succeed for the first time in training aconvolutional network to jointly estimate the depth and thecamera motion from an unconstrained pair of images.

unconstrained

variational

approximations

arguablely

propagate

aggregate

farthest

translation-invariant

points that is irregular andorderless

(1) point feature learn-ing, (2) point mixture, and (3) flow refinement.

Figure 2

given P and Q

XY Z only

key contributions

fine tuning

generalizability

our network

given input point clouds from two consec-utive frames (point cloud 1 and point cloud 2), our networkestimates a translational flow vector for every point in thefirst frame to indicate its motion between the two frames

(7)

(6)

mitigate

the parallelization

Although this method seems very simple, itworks surprisingly well as shown in the results

Piecewise processing

In-frame motion compensation

cannot be constantly tracke

we employthe LiDAR reflectivity as the 4-th dimensional measurement

local smoothness

Fig. 6

To increase the localization and mapping accuracy, weremove any of the following points

(4)

Small intensityI(P)

Fig. 5

Our con-tributions are

revious work were based on spinningLiDARs,

better performance but cannot run in real-time

linear interpolatingthe LiDAR pose

[12] , [17] and [18]

key-points based method

generalized ICP

ICP algorithm

new features

This regular scanning greatly simplifies the featureextraction

linear (et) and angular (er) errors

Fig. 9. Statistics from real sensor data: Dispersion in the localization of thereference points (a) and calibration deviation from the final result (linear, inm, and angular, in rad) (b)

Table VI shows the linear (et) and angular (er) calibrationerrors

2)

s the distance increases

Experimentation in the synthetic suite can be divided into threedifferent focus points:

Gazebo simulato

except for the tuning of the pass-through filters mentionedin Sec. IV,

without user interven-tion

n the one hand,

On the other hand

Umeyama registration procedure

Distances from this point to the other three determine thecorrect ordering

Note thatthis condition would not be fulfilled when calibrating a front-facing 360° LiDAR scanner and a rear-looking camera, forinstance

that is, pairs of points representing the samereference points in both clouds must be associated

The goal of the registration step is to find the optimaltransformation parametersˆθso that when the resulting trans-formationˆTis applied

2)

he time necessary to completethe procedure depends on the sensor’s framerate but is rarelylonger than a few seconds.

Ndata frames

To increase the robustness of thealgorithm,

requires the detection ofArUco markers,

Pp

2D circle segmentation isused to extract a model of the pattern holes present inP4

XY-plane coincides withπand projecting allthe 3D points ontoπ

P3contains only points representing the edges of the calibrationtarget; that is, the outer borders and the holes.

impose that the plane must be roughly vertical in sensorcoordinates

RANSAC is applied toP1

The intended outcome is an estimate ofthe 3D location of the centers in sensor coordinates.

PS

Sobel filter is applied over the image

PS1

the calibration target isexpected to have some texture

PS0can be straightforwardly obtained

PL2

same scan plane

(1)

PL1

pass-through filters are applied in the threecartesian coordinates to remove points outside the area wherethe target is to be placed

The monocularalternative is substantially different as it relies on the ArUcomarkers instead

In all cases, the output ofthis stage is a set of four 3D points representing the center ofthe holes in the target, in local coordinates

hree different variants of the procedure areproposed here, one per sensor type

localization of

registration

thesegmentation

The proposed calibration algorithm, illustrated in Fig. 3

As noticeable in the two different embodiments shown inFig. 2,