• Russia
• 1 Year ago
• Offline
• 1 Year ago

About me

This article was written by xinyang guo and baoyu han, bigo engineers, and translated by rosie zhang. The bigo (bigo) technology is singapore's first extremely fast growing device. Companies. Bigo video products and services based on ai technology have earned strong recognition in the world: over 400 many thousands of people in other than 150 countries. Standards include bigo live (live broadcast) and likee (short video). Likee is a global resource for launching short videos where passengers can share their own moments, express their views and hang out with the outside world. . In order to improve contact with the customer and recommend more unique content to users, likee it is necessary to weed out duplicate entries from the many videos created by users daily, which is not a complete task. This blog hints, as well as bigo uses milvus, a vector knowledge base with free primary code, to effectively remove duplicate videos. Go to: Overviewdeduplication workflow videosystem architectureusing milvus for similarity matching Milvus - includes a vector detail archive with free primary code, providing ultra-fast vector searches. Likee from milvus should prepare the lookup within 200ms, while still providing a fast recall speed. Meanwhile, by horizontally scaling milvus, likee is successfully increasing vector query throughput, increasing its performance to a greater extent. Video deduplication workflow How likee identifies duplicates video ? Whenever a video request is screwed into the likee network, it will be cut into 15 percent percent percent percent-20 shot and each frame will be converted into a feature vector. Likee then searches the knowledge bank of 700 million vectors to collect the k most similar vectors. Any of the higher k vectors corresponds to a movie in the registry. Likee additionally performs refined searches to get reliable effects and choose which videos to remove. System architecture Let's take a closer look at how likee's video is removed. -Duplication system deals with milvus. As shown in the diagram below, new videos uploaded to likee will be written to kafka, the information storage system, in the rhythm of the current hours and days, and consumed by kafka consumers. The feature vectors of these videos are extracted using deep learning models, where the unstructured videos are converted into feature vectors. These feature vectors will be packaged by the system and sent to the similarity auditor. The extracted feature vectors will be indexed by milvus and stored in ceph, youtube video to gif imgur before being loaded by the milvus query node for prospective search. The corresponding video ids of these feature vectors will also be in tidb or pika at the same time, according to actual needs. Using the milvus vector registry for similarity searches there are billions of existing ones when looking for similar vectors data, together with the large volumes of material generated on a daily basis, form a huge circumstance for the functionality of the vector search engine. As a result of extensive analysis, likee finally chose milvus, any distributed vector browser with improved performance and excellent recall speed, to accommodate vector similarity surfing. As shown in the diagram below, the similarity search procedure is performed like this: 1. First of all, milvus does a batch search to find the 100% most similar vectors for any of the variety of feature vectors extracted from new porn. Each similar vector is associated with the corresponding video id. 2. Further, by comparing video ids, milvus removes duplicate sex files, and extracts feature vectors of the remaining files from tidb or pika. 3. Finally, milvus calculates and evaluates the similarity between each set of extracted secret vectors and the feature vectors of the video query. The id of the content with the best score is returned as the result. So video similarity search is over. As a high-performance vector search engine, milvus has done an incredible job in the likee video deduplication network, greatly spurring the growth of bigo short videos. Business. And in the video business, there are many other scenarios where milvus can be used, such as blocking illegal text or authoritative video advice. Both bigo and milvus look forward to a long-term partnership in other regions.

Social Links

• http://https://getof.net/imgur-downloader/
• n/a
• n/a
• n/a