Home News Community

Machine Learning Meets Jobsite Photos


Photo and video documentation is exploding in construction with ever-increasing amounts being created from mobile devices, drones, and new devices on the jobsite. Except for specific workflows (images attached to reports, issues, or plans), many photos and videos wind up on shared drives or on the hard drives of field personnel. Every project manager has a shoebox or drawer full of USBs and hard drives with their old data stored on them.  Your IT team may have gigabytes or terabytes of storage taken up by old photo and video data.  

Once you get your hands on the files, you just run into the next set of problems. Finding critical project photos and video documentation is a major challenge as file names are not readable or helpful for searching, and renaming every file takes too much time. But the content within each is critical to the project. It’s vital for resolving conflicts with trades, showing progress to the owner, establishing best practices, defending against claims, and much more. 

With ongoing advances in the world of machine learning, solving the challenges of video and photo management and search is getting easier every day.   

Now there’s a solution via Machine Learning. If you’ve ever searched for a friend in your photos on Facebook, for example, or used Google to find specific images online, then you’ve already used computer vision, which is a type of machine learning. If you’ve ever asked a question to Apple’s Siri then you’ve already used speech recognition, another type of machine learning. 

Now you have the ability to apply these techniques to your current and old project photos and videos. Here are some important questions machine learning can answer: 

Q: How many images do you have of people on your projects in unsafe situations (like missing a hard hat or other PPE)? Hopefully it is zero, but how can you make it easy to find and fix these problems?
A: Search your pictures for “people” or “hard hat” to locate all imagery that you need to review.

Q: You’re looking at a bill and need to verify that you were charged correctly. When was the last time a crane was on the project? Or how much scaffolding was in place at the end of last month?  
A: Search your pictures for “crane” or “scaffolding” and see the last image in a date and time-stamped list.

Q: What’s the best image to send to marketing? Marketing is asking for some “hero shots” to use in an upcoming proposal.
A: Search for “building” and then tag good images “for marketing use.” Share a link with them to access only those photos in your project set. 

Q: What’s the history of field observations for the main lobby (or other location), including daily photos and milestone documentation for in-wall, above-ceiling, etc?
A: Search field videos (or voice annotated photos) for key tags that relate to that location and bring them up in a timeline. Bonus feature: Create a link to that search query,  paste it into your plan or Building Information Model (BIM) to access “always up to date” photo and video data for that space. 

What’s next for these capabilities? 

As a first step, they are now more closely aligned with your workflows in Procore through an integration in the Procore App Marketplace. By linking automatically to Procore, these photos can now appear in your Photos-Albums area and then be used as attachments for any of your Procore workflows, like your Daily Logs, Drawings, Punch Lists, Inspections and more. You can also now automatically search any images or videos you take using these machine learning generated tags because the tags appear as photo descriptions in Procore. And there is more to come. With ongoing advances in the world of machine learning, solving the challenges of video and photo management and search is getting easier every day.  


Add New Comment