參加人數
Organizer
Pervasive Artificial Intelligence Research (PAIR) Labs, National Chiao Tung University (NCTU), Taiwan
The Pervasive AI Research (PAIR) Labs, a group of national research labs funded by the Ministry of Science and Technology, Taiwan, is commissioned to achieve academic excellence, nurture local AI talents, build international linkage, and develop pragmatic approaches in the areas of applied AI technologies toward services, products, workflows, and supply chains innovation and optimization. PAIR is constituted of 18 distinguished research institutes in Taiwan to conduct research in various of applied AI areas.
Website: https://pairlabs.ai/
Industrial Partner
MediaTek
MediaTek Inc. is a Taiwanese fabless semiconductor company that provides chips for wireless communications, high-definition television, handheld mobile devices like smartphones and tablet computers, navigation systems, consumer multimedia products and digital subscriber line services as well as optical disc drives. MediaTek is known for advances in multimedia, AI and expertise delivering the most power possible – when and where needed. MediaTek’s chipsets are optimized to run cool and super power-efficient to extend battery life. Always a perfect balance of high performance, power-efficiency, and connectivity.
Web Site: https://www.mediatek.com/
Academic Partner
Intelligent Vision System (IVS) Lab, National Chiao Tung University (NCTU), Taiwan
The Intelligent Vision System (IVS) Lab at National Chiao Tung University is directed by Professor Jiun-In Guo. We are tackling practical open problems in autonomous driving research, which focuses on intelligent vision processing systems, applications, and SoC exploiting deep learning technology.
Web Site: http://ivs.ee.nctu.edu.tw/ivs/
Wistron-NCTU Embedded Artificial Intelligence Research Center
Sponsored by Wistron and founded in 2020 September, Wistron-NCTU Embedded Artificial Intelligence Research Center (E-AI RDC) is a young and enthusiastic research center leaded by Prof. Jiun-In Guo (Institute of Electronics, National Chiao Tung University) aiming at developing the key technology related to embedded AI applications, ranging from AI data acquisition and labeling, AI model development and optimization and AI computing platform development with the help of easy to use AI toolchain (called ezAIT). The target applications cover AIoT, ADAS/ADS, smart transportation, smart manufacturing, smart medical imaging, and emerging communication systems. In addition to developing the above-mentioned technology, E-AI RDC will also collaborate with international partners as well as industrial partners in cultivating the talents in the embedded AI field to further enhance the industrial competitiveness in Taiwan Industry.
Dear Participants, The ACM ICMR 2021 Grand Challenge: PAIR competition awardees are: Champion: "as798792" (國立台南大學) 1st Runner-up: "Deep Learner" (北京交通大學) 3rd-place: "UCBH" (國立成功大學電通所) Best accuracy award: "as798792" (國立台南大學) Best bicycle detection award: "as798792" (國立台南大學) Best scooter detection award: "abcda" (國立臺灣科技大學) Congratulations!
Dear Participants, The ACM ICMR 2021 Grand Challenge: PAIR competition awardees are: Champion: "as798792" (國立台南大學) 1st Runner-up: "Deep Learner" (北京交通大學) 3rd-place: "UCBH" (國立成功大學電通所) Special Awards Best accuracy award: "as798792" (國立台南大學) Best bicycle detection award: "as798792" (國立台南大學) Best scooter detection award: "abcda" (國立臺灣師範大學資工系) Congratulations!
ivslab_test_qualification.tar has been released, please download it and upload your submission.csv
Object detection in the computer vision area has been extensively studied and making tremendous progress in recent years using deep learning methods. However, due to the heavy computation required in most deep learning-based algorithms, it is hard to run these models on embedded systems, which have limited computing capabilities. In addition, the existing open datasets for object detection applied in ADAS applications usually include pedestrian, vehicles, cyclists, and motorcycle riders in western countries, which is not quite similar to the crowded Asian countries like Taiwan with lots of motorcycle riders speeding on city roads, such that the object detection models training by using the existing open datasets cannot be applied in detecting moving objects in Asian countries like Taiwan.
In this competition, we encourage the participants to design object detection models that can be applied in Taiwan’s traffic with lots of fast speeding motorcycles running on city roads along with vehicles and pedestrians. The developed models not only fit for embedded systems but also achieve high accuracy at the same time.
This competition is divided into two stages: qualification and final competition.
Qualification competition: all participants submit their answers online. A score is calculated. The top 15 teams would be qualified to enter the final round of the competition.
Final competition: the final score will be evaluated on MediaTek Dimensity 1000 Series platform for the final score.
The goal is to design a lightweight deep learning model suitable for constrained embedded system design to deal with traffic in Asian countries like Taiwan. We focus on detection accuracy, model size, computational complexity, performance optimization and the deployment on MediaTek’s Dimensity 1000 platform.
With MediaTek’s Dimensity 1000 platform and its heterogeneous computing capabilities such as CPUs, GPUs and APUs (AI processing units) embedded into the system-on-chip products, developers are provided the high performance and power efficiency for building the AI features and applications. Developers can target these specific processing units within the system-on-chip or, they can also let MediaTek NeuroPoint SDK intelligently handle the processing allocation for them.
Given the test image dataset, participants are asked to detect objects belonging to the following four classes {pedestrian, vehicle, scooter, bicycle} in each image, including class, bounding box, and confidence.
According to the points of each team in the final evaluation, we select the highest three teams for regular awards.
Champion: $USD 1,500
1st Runner-up: $USD 1,000
2nd Runner-up: $USD 750
Special Awards
Best accuracy award – award for the highest mAP in the final competition: $USD 200;
Best bicycle detection award – award for the highest AP of bicycle recognition in the final competition: $USD 200;
Best scooter detection award – award for the highest AP of scooter recognition in the final competition: $USD 200;
All the award winners must agree to submit contest paper and attend the ACM ICMR2021 Grand Challenge PAIR Competition Special Session to present their work.
Date | Activity |
---|---|
1/4/2021 | Qualification Competition Start Date |
1/4/2021 | Date to Release Public Testing Data |
3/1/2021 | Date to Release Private Testing Data for Qualification |
3/8/2021 12:00 PM UTC | Qualification Competition End Date |
3/9/2021 12:00 AM UTC | Finalist Announcement |
3/9/2021 | Final Competition Start Date |
3/15/2021 | Date to Release Private Testing Data for Final |
3/22/2021 12:00 PM UTC | Final Competition End Date |
4/12/2021 12:00 PM UTC | Award Announcement |
Qualification Competition
The grading rule is based on MSCOCO object detection rule.
The mean Average Precision (mAP) is used to evaluate the result.
Intersection over union (IoU) threshold is set at 0.5.
Model size should be less than 200MB.
The resulting average precision (AP) of each class will be calculated and the mean AP (mAP) over all classes is evaluated as the key metric.
Besides, during the qualification competition period, each team has to submit a team composition document, including team name, leader, team members, affiliation, and contact information, etc.
Final Competition
Accuracy of final submission cannot be 5% lower (include) than their submitted model of qualification.
The summation of Preprocessing & Postprocessing time of final submission cannot be 50% slower (include) than the inference time of the main model. (evaluated on the host machine)
The team with the highest accuracy will get the full score (40%) and the team with the lowest one will get zero. The rest teams will get scores directly proportional to the mAP difference.
The team with the smallest model will get the full score (15%) and the team with the largest one will get zero. The rest teams will get scores directly proportional to the model size difference.
The team with the smallest GOP number per frame will get the full score (15%) and the team with the largest one will get zero. The rest teams will get scores directly proportional to the GOP value difference.
The team with a single model (w/o Preprocessing & Postprocessing) to complete the detection task in the shortest time will get the full score (30%) and the team that takes the longest time will get zero score. The rest teams will get scores directly proportional to the execution time difference.
The evaluation procedure will be toward the overall process from reading the private testing dataset in final to completing submission.csv file, including parsing image list, loading images, and any other overhead to conduct the detection through the testing dataset.
Reference
[1] Average Precision (AP):
https://en.wikipedia.org/wiki/Evaluation_measures_%28information_retrieval%29#Average_precision
[2] intersection over union (IoU):
https://en.wikipedia.org/wiki/Jaccard_index
[3] COCO API:
https://github.com/cocodataset/cocoapi
Ted Kuo, tkuo@nctu.edu.tw
Jenq-Neng Hwang, hwang@uw.edu
Jiun-In Guo, jiguo@nctu.edu.tw
Po-Yu Chen, marvin.chen@mediatek.com
Hsien-Kai Kuo, hsienkai.kuo@mediatek.com
Chia-Chi Tsai, chia-chi.tsai@mediatek.com