MobilTech (Seoul, KR)
Method of collecting information about road signs using the mobile mapping system which can collect information on the markings on the road in real time and convert the information onto an extremely high-definition map using the mobile mapping system comprising a remote sensing device as well as a navigation device, is described. In particular, the system is able to capture accurate road marking information in real-time by separating objects and attributes from multiple images based on deep learning and map the road marking information to a high-definition map, and remarkably improve accuracy of recognizing the road marking information. Therefore, performance and reliability could be improved in the field of autonomous driving and the field of road recognition and the high definition road map field for autonomous driving, and the fields similar or related thereto.
Field of the Invention
The invention is a method for gathering information on road signs using an app for mapping on smartphones. In particular, it is a method for collecting information on road signs using a smartphone mapping system. This system can collect information on road markings in real time, then map them onto a high-definition map, and use a combination of remote sensing devices and a navigation device using a handheld mapping system.
The present invention is particularly pertinent to a method of gathering information on road signs with mobile mapping systems. This is done by extracting objects from successive images on the basis deep learning. It can also map the road markings information onto high definition maps, and greatly increase the accuracy in recognising the road markings information.
Background of Related Art
Unmanned autonomy of vehicles (autonomous vehicle) generally consists of planning a driving route in accordance with the surroundings and understanding the environment.
High definition maps (HD-maps) are three-dimensional maps that include information on the surroundings like roads, terrain elevations and curvatures. The HD-map also contains the information needed to drive on roads.
For instance, the HD-map includes various information needed for driving on the road in question, such as lanes, driving directions intersections, traffic signals, traffic lights speed limits, and other pertinent information for the road.
A vehicle that is autonomous can drive on the road and recognize surrounding environments based on the HD-map. This technique is called Road Information Detect Method for Auto-Driving Robot’ autonomous driving robotic using TheSame that is described in Korean Patent Registration Number. This is a prior art document that is described below.
As described previously, it is claimed that the HD-map is crucial in the field of autonomous driving, and also requires most correct and rapid update to ensure safe autonomous driving.
The HD-map is generally created through four steps: planning work, data acquisition , and point cloud data generation. Editing and drawing objects are also included.
Within the four stages in the process, the one of data acquisition gathers information about the road mostly by using a driving device (vehicle or something similar) equipped with LiDAR or similar as well as three-dimensional point cloud information of the road can be generated by the data gathered.
Because the three-dimensional point cloud data is only a snapshot about the roads’ shapes and shapes, it is essential to include information that can be accessed by a driver traveling on the road. This is information such as roads’ markings on the surface, traffic signs and traffic lights.
However, since the majority of this information is manually mapped to a HD-map, there is a problem in that time and cost are required to create the HD-map.
(Patent Document 1) Korean Patent Registration No. 10-1348941 “Road information detection method for auto-driving robotic robot as well as auto-driving bot utilizing The Same”
Therefore, the present invention has been made to address the above problems, and it is an object of this invention to provide a method of collecting road sign information using a mobile mapping system, which can collect information onthe markings on the road in real time and convert the information into a high-definition map employing a mobile mapping system comprising a remote sensing device and the navigation device.
A further object of this invention is the creation of a method for collecting road markings information through mobile mapping systems. This can be done by separating objects from successive images using deep learning. This will and mapping the road markings to high-definition maps and increase the accuracy in recognizing road markings.
Furthermore, one of the objectives of the present invention is the provision of collecting road sign information through a mobile mapping device that can dramatically improve recall and precision (accuracy) in recognising an object as well as attribute information through a process of cleansing a result of recognizing the object as one result.
To accomplish the above-mentioned objectives an approach to collecting road sign information using a mobile mapping system according to the embodiment of the invention includes the following: a collection step that collects road data comprising point cloud data and image data by using the mobile mapping system and an object recognition process of detecting a bounding area for each object that is included in the successive image data and then confirming the object’s characteristics of the object; and an object mappingstep of calculating the spatial coordinates of a similar object in a high-definition map from the point cloud data.
The object recognition step can include an affirmation step for boundingbox data, which confirms information about bounding boxes that include location coordinates for each image as well as object attribute information for a corresponding boundingbox and a collection process for boundingbox informationthat tracks the coordinates of the location for each boundingbox and collects details about the boundingbox of each image; and an object attribute determination that selects the object with the highest number of boundingboxes among objects that share the same attribute information in all boundingboxes tracked as the object of the object the object the object of the object the object the object the object of the object the object the object, or as the object of the object of the object of object of object of object of the object of object of object of the object
In addition, at the bounding box information verification stage, information regarding the bounding boxes that are sequentially collected for each image may be saved on two-dimensional vector.
Additionally, during the bounding-box information gathering step, the steps of confirming the location coordinates for the same binding box from the current data, and then verifying a box in close proximity to the same coordinates in the next image can be performed in a sequence. This lets you follow the bounding boxes that are adjacent to the respective bounding boxes for every image.
Furthermore, at the bounding box information collection step, there will be one bounding area in the following image data that is within the distance of a tracking setting from an identical location in the present image data based off of the locationcoordinates for the bounding box that is corresponding, the corresponding bounding box can be chosen.
Furthermore, during the bounding box information collection step, there must be at minimum two bounding boxes within the following image data that are within a tracking setting distance from an equivalent position in the current image data on the basis of the location coordinates of the bounding box, the closest bounding box could be chosen.
In addition, at the information collection for bounding boxes step, if there is no bounding box to exist in the subsequent image data within the distance of a tracking setting from the same location in the current image data using the locationcoordinates of the corresponding bounding box, the tracking of the corresponding bounding box may be put on hold.
Furthermore, during the information collection for bounding boxes step, if there is a predetermined number of image data are not have a bounding boxes in the following image data that is within a tracking setting distance from the current image data based off of the location coordinates of the corresponding bounding box is accumulated, the tracking of the appropriate bounding box could be stopped.
In addition, the object mapping step may include: a point cloud data extraction step of extracting point cloud data in an inner area of a bounding box of each object from the point cloud data; a coordinate system conversion step of convertingthree-dimensional coordinates of the point cloud data into a map coordinate system of a high definition map; and an object coordinate calculation step of calculating coordinates of an object included in the corresponding bounding box from the convertedmap coordinates.
The median value of the converted points from the point cloud data could be calculated in the point of calculation for object coordinates. The median value could then be used to calculate the coordinates for each object that is within the bounding boxes.
Click here to view the patent on USPTO website.
Get Patents with PatentPC
What is a patent?
Granted by the government to safeguard an invention, patents give the inventor with the exclusive right to create, use and sell the invention?society is benefited when a brand new technology is introduced to the market. These benefits could be realized directly when people can accomplish feats previously unattainable and indirectly by the economic opportunities which innovation can bring (business growth, employment).
Patent protection is sought by a variety of pharmaceutical companies and university researchers for their research and development. Patents can be granted for products, processes, or method of making new materials. Patent protection is granted to any invention that is valuable or novel and is not already known by others in the same area.
Patents honor inventors who have commercially profitable inventions. They provide a reason for inventors to invent. Patents allow inventors and small businesses to know that there is the possibility that they’ll be paid back for their time, effort, and money invested in technological development. They could earn a decent income from their work.
Companies that are able to:
Protect innovative products and services;
Increase the visibility and value of your products on market
Differentiate yourself and your products from the rest.
Find out about business and technical information.
Beware of the possibility of accidentally using third-party proprietary content, or losing your important information, creative outputs, or other creative output.
Patents effectively transform the inventor’s information into a tradeable asset which opens new opportunities for job creation and expansion of businesses by licensing or joint ventures.
Small companies that have patent protection will be more attractive to investors involved in the commercialization and development of technology.
Patenting can lead to new ideas and inventions. The information you create may be eligible for patent protection.
Patents are a way to stop untrustworthy third-party companies from earning from the invention’s efforts.
Patent-protected technology that is commercially viable can be used to finance technological research and development (R&D) which increases the likelihood of a better technology in the near future.
You can use intellectual property ownership to convince investors and lenders that your product has real commercial value. A single patent could lead to multiple financing opportunities. Patents can be used in conjunction with other IP assets as collateral or security to secure financing. Investors may also look at your patent assets to increase their company valuation. Forbes and other sources have stated that every patent can boost the value of your company by as much as $500,000 to $1 million.
A well-constructed business plan is essential for start-ups. It must be built on IP and show what your service or product is distinct. Investors will also be impressed if you demonstrate that your IP rights are secure or is in the process of becoming secure, and that they align with your business plan.
It is important to keep your invention secret until you submit for patent protection. It is important to keep an invention secret prior to making a patent application. Public disclosure can often ruin the novelty of the invention, making it inepatentable. Thus, disclosures that are filed prior to filing (e.g. for testing marketing investors, test-marketing, or any other business partners) must only be done upon signing a confidentiality contract.
There are a variety of patents. Understanding them is crucial to protect your invention. Utility patents protect new processes and machine creations. Design patents cover ornamental designs. Utility patents are the best as they protect the proprietor from copycats and other competition. Utility patents are often granted to enhance or modify existing inventions. Patents issued under utility can be used to enhance or alter existing inventions. A process patent could describe the methods or actions to perform a particular action. But, a chemical composition would include an amalgamation of components.
What is the average length of a patent? Patents for utility last for 20 years after the initial filing date, but their expirations can be extended due to delays at the patent office for instance.
Are you interested in patenting your ideas? Patents are only granted to applicants who are the first to file, which is why you must file as quickly as possible. Contact PatentPC now to have your patent application submitted!
When you are writing a patent application when you are writing a patent application, it is important to conduct a patent search, as the search will give you an understanding of other people’s concepts. This can help you limit the scope of your idea. You can also find out about the latest developments in your area of invention. You’ll be able to get a better idea of what your invention ought to be, and you’ll be more prepared to submit the patent application.
How to Search for Patents
The first step to get the patent you want is to conduct a patent search. You can do a google patent search or do a USPTO search. Patent-pending is the term for the product protected by the patent application. You can search the public pair to locate the patent application. Once the patent office approves the patent application, you can conduct a patent number search to find the issued patent and your product has been granted patent. You can also use the USPTO search engine. See below for details. You can get help from an attorney who specializes in patents. Patents in the United States are granted by the US trademark and patent office or the United States patent office. The trademark office also evaluates trademark applications.
Are you interested in similar patents? These are the steps:
1. Create a list of terms for your invention in relation to its intended, composition, or use.
Write down a concise detailed explanation of your invention. Be sure to avoid using terms that are generic such as “device,” “process,” and “system.” Instead, consider synonyms to the terms you selected initially. Next, note important technical terms and key words.
Use the questions below to help you find keywords or concepts.
- What is the objective of the invention? Is it a utilitarian device or an ornamental design?
- Is the invention a method of creating something or performing an action? Is it a product or procedure?
- What is the composition and function of the invention? What is the physical composition of the invention?
- What’s the objective of the invention?
- What are technical terms and phrases that define the essence of an invention? To find the right terms, refer to the technical dictionary.
2. Use these terms to find pertinent Cooperative Patent Classifications for your invention at the Classification Text Search Tool. If you’re unable to locate the appropriate classification for your invention, scan through the classification’s class Schemas (class schedules) and try again. If you do not get results from the Classification Text Search, you might consider substituting your words that describe your invention using synonyms.
3. Check the CPC Classification Definition for confirmation of the CPC classification you have found. If the selected classification title includes a blue box that contains “D” and “D”, then the hyperlink to the CPC classification definition will be provided. CPC classification definitions will help identify the specific classification’s scope which is why you can be certain to pick the most pertinent. They may also provide research tips or other suggestions that can be useful for further study.
4. Get patent documents using the CPC classification from the Patents Full-Text and Image Database. You can look through and select the most relevant patent publications focusing first on the abstract and the drawings that are representative.
5. Take advantage of this list of most pertinent patent documents to look at each one in depth for any the similarities to your idea. Pay attention to the specification and claims. It is possible to find additional patents by referring to the patent examiner as well as the applicant.
6. Retrieve published patent applications with the CPC classification you chose in Step 3 of the Applications Full-Text and Image Database. It is also possible to use the same method of search that you used in step 4 to limit down your search results to the most relevant patent applications by looking over the abstracts and drawings for every page. After that, take a close look at the published patent applications and pay particular attention to the claims and the additional drawings.
7. You can search for other US patent publications using keywords searching in the AppFT and PatFT databases, as well as the classification search of patents that are not from the US according to below. Also, you can utilize web search engines to find non-patent literature disclosures about inventions. Here are a few examples:
- Add keywords to your search. Keyword searches may turn up documents that are not well-categorized or have missed classifications during Step 2. For example, US patent examiners often supplement their classification searches with keyword searches. Think about the use of technical engineering terminology rather than everyday words.
- Search for foreign patents using the CPC classification. Then, re-run the search using international patent office search engines such as Espacenet, the European Patent Office’s worldwide patent publication database of over 130 million patent publications. Other national databases include:
- European Patent Office (EPO) provides esp@cenet to access a network of Europe’s patent databases with access to machine translation of European patents.
- Japan Patent Office (JPO) – with access to machine translations of Japanese patents.
- World Intellectual Property Organization (WIPO) offers PATENTSCOPE with a full-text search of published international patent applications and machine translations for some documents, as well as a list of international patent databases.
- Korean Intellectual Property Rights Information Service (KIPRIS)
- State Intellectual Property Office (SIPO) with machine translation of Chinese patents.
- Other International Intellectual Property Offices with online patent databases include Australia, Canada, Denmark, Finland, France, Germany, Great Britain, India, Israel, Netherlands, Norway, Sweden, Switzerland, and Taiwan.
- Search non-patent literature. Inventions can be made public in many non-patent publications. It is recommended that you search journals, books, websites, technical catalogs, conference proceedings, and other print and electronic publications.
To review your search, you can hire a registered patent attorney to assist. A preliminary search will help one better prepare to talk about their invention and other related inventions with a professional patent attorney. In addition, the attorney will not spend too much time or money on patenting basics.