Takeshi Ishita

Work Experience

CyberAgent AI Lab

R&D of image recognition and localization for retail store environments

  • R&D of a real-time monocular camera localization method that leverages a map built with COLMAP

  • Implementation of an odometry estimation method using the RealSense stereo infrared camera

  • R&D of an in-store product search method using CLIP, a multimodal foundation model

  • Development of a tool that automates environment setup on the in-house compute platform for machine learning

July 2024 -

Tier IV, Inc.

R&D of vehicle localization

  • Survey and implementation of Visual SLAM techniques

  • Development of localization algorithms for challenging environments

  • Implementation of a LiDAR driver and a localization method running on Autoware Kernel, Tier IV's in-house real-time OS kernel

July 2020 - June 2024

Mitou Program

Development of a Visual SLAM framework

April 2019 - March 2020

DeNA Co., Ltd.

Survey and implementation of computer vision methods

April 2018 - March 2020
Part-time job

Cookpad Inc.

Design and implementation of machine learning methods for ingredient recognition from food images.

Dec 2016 - Jul 2017
Part-time job

Usagee Inc.

  • Research and development of Machine Learinng & Computer Vision methods

  • Providing effective solutions to customers

May 2014 - Jan 2017
Part-time job

Education

National Institute of Technology, Tokyo College, Advanced Course
April 2017 - March 2019
Student exchange with Metropolia University of Applied Sciences
August 2017 - December 2017
All courses were taken in English. In a joint project with local students, I implemented an algorithm that evaluates the reliability of user posts for Huoletti, a healthcare-related posting app.
National Institute of Technology, Tokyo College
April 2012 - March 2017

Patent

Patent #6306770 "Information processing apparatus, information processing method, and program" (JP 2018-181267 A)

A method that estimates the ingredients used in a dish from its photo, combining deep learning with a graph search algorithm.
Inventor: Takeshi Ishita (sole inventor) / Applicant: Cookpad Inc.
Filed on April 21, 2017 / Granted on April 4, 2018

Awards and Contests

  • JPHACKS 2015, finalist

  • Open Hack U 2014, participant

  • SECCON 2013, participant

My works

My works are available on GitHub

Visual Relocalization

Camera localization on a COLMAP map using a monocular camera.

_images/demo-relocalization.gif

icp_rust

An implementation of ICP (Iterative Closest Point) in Rust.
It is intended to run on the OS being developed at Tier IV, but it also runs on Linux.
_images/icp.gif

YDLiDAR Driver

A Rust driver for YDLiDAR T-mini Pro, a palm-sized LiDAR.

https://raw.githubusercontent.com/tier4/ydlidar_driver/main/images/plot_scan.gif

Velodyne Driver

A Rust driver for the Velodyne VLP-16 LiDAR.
It is intended to run on Autoware Kernel, the real-time OS kernel developed at Tier IV, and was written from scratch based on the device specification.

LOAM (Refactored version)

A refactored version of LOAM [7] [8], a LiDAR-based SLAM algorithm.
It is intended to run integrated with Autoware, an open source software for autonomous driving.
The structural feature extraction from LiDAR scans is several times faster than the original implementation.

Tadataka

A package that implements several Visual Odometry algorithms. It offers both an odometry estimation algorithm for RGB-D cameras and a feature-based odometry estimation system for monocular cameras.

DVO (Dense Visual Odometry) [10] [3]

Estimating camera motion from depth and image intensity.

Feature Based Visual Odometry

Estimating a 3D map and the camera trajectory from a monocular RGB image sequence alone.


DVO

A reimplementation of Dense Visual Odometry [10].
It estimates odometry from RGB-D camera input by directly minimizing pixel intensity errors. I implemented it to learn the theoretical background of this method, which laid the foundation of dense Visual Odometry.

RoadDamageDetector

_images/road-damage-1.png
Road damage detector based on SSD (Single Shot Multibox Detector) [4].
It reproduces the work of Maeda et al. [6]. The detailed explanation is at my Qiita blog page (in Japanese).
I implemented two base networks, VGG16 [9] and ResNet-101 [2], and compared their accuracy and runtime. VGG16 reached mAP 0.561 while ResNet-101 reached mAP 0.379, but on CPU, ResNet-101 ran more than twice as fast as VGG16.

SBA

A Python implementation of Sparse Bundle Adjustment (SBA) [5], an efficient 3D reconstruction algorithm.
The original implementation does not make the derivation of the Jacobian explicit in code, which makes the algorithm hard to follow. I wrote a reimplementation that maps the overall architecture and the Jacobian computation to functions in the code as clearly as possible.

Tomasi-Kanade

_images/tomasi-kanade-output-2.png
An implementation of 3D reconstruction by the Tomasi-Kanade method [11].
It assumes an orthographic camera model, which limits its use in real-world applications, but it can reconstruct 3D structure quickly via SVD.

Bilinear-Interpolation-SIMD

An implementation of bilinear interpolation using SIMD. A plain C implementation of the same method and benchmark scripts for speed comparison are included.

PCANet

_images/pcanet.png
PCANet [1] is a neural network for image classification that trains its weights with PCA, which makes batch training fast. Its pooling layer requires histogram calculation, and that was the bottleneck of the execution speed.
To resolve this, I implemented the histogram calculation for GPU in CuPy, which sped it up substantially. With the help of the CuPy developers, it was later merged into CuPy itself. #298

Ensemble PCANet

PCANet consists only of linear operations, so its representation ability is limited. I treated PCANet as a weak learner and applied ensemble learning (Bagging) to improve classification performance. On MNIST, it reached 94.47% accuracy against 92.02% for the original PCANet.
This idea was presented at JSAI 2017.

DTrie

An implementation of the succinct data structure Trie in D.
I implemented it to hold a Kana-Kanji conversion dictionary in a small amount of memory.

SCW

Implementation of SCW (Soft Confidence-Weighted Learning) [12].
SCW is an online supervised learning algorithm which utilizes all the four salient properties:
  • Large margin training

  • Confidence weighting

  • Capability to handle non-separable data

  • Adaptive margin

Blog

The titles below are given in the original Japanese, followed by an English translation in parentheses.

Article

Conference Presentations

Presentations

References