The PG&E Disaster and the Case for Digitizing Physical Documents

The IT industry purely loves digital information and with good reason. Creating, storing and archiving documents consisting of 1’s and 0’s provides the foundation for thousands of business computing solutions and billions of dollars in annual sales.

While the gospel of the “all-digital enterprise” inspires many proselytizers and true believers, companies in the real world tend to utilize both digital and physical documents and records. However, that process can result in an uneasy balancing act with sometimes serious repercussions.

Want an extreme worst-case example? Look no further than Pacific Gas and Electric (PG&E), the utility company that provides electric and natural gas services to most of Northern and Central California.

On September 9, 2010, a PG&E natural gas transmission pipeline in San Bruno ruptured and exploded into a devastating fire, killing eight people and destroying 38 homes. Not surprisingly, this tragic loss of life and property led to multiple investigations by PG&E, local authorities, insurance carriers and the California Public Utilities Commission (CPUC).

Those efforts intensified when it was discovered that though the company’s records described the pipeline as seamless, investigators found multiple poorly welded seams that they believed might have caused or contributed to the rupture and blast. Complicating matters further, PG&E had never conducted inspections on the line that were capable of discovering bad welds.

Paper Chase

As a result, in October 2010, the CPUC ordered PG&E to produce records detailing the history, construction, performance and condition not just of the San Bruno pipeline but of its entire gas transmission infrastructure by March 15, 2011.

One might think that complying with such a sizable order wouldn’t be that a big deal for a modern utility company driving nearly US$14 billion in annual revenues. One might think that, but one would be seriously wrong.

PG&E’s efforts to digitize its records and archives reportedly have been troubled for years. So, meeting the CPUC’s order resulted in a paper chase of monumental size. A preliminary study suggested that the company’s pipeline archive included around 1.2 million physical documents, including inspection records, studies and reports.

PG&E rented San Francisco’s Cow Palace, a cavernous venue originally designed to host livestock shows and rodeos, and commenced hand searching nearly 100,000 file boxes of paper records.

The sheer impact of that many physical file boxes is hard to grasp, so what does it mean in practical terms? The boxes — which are stored in warehouses near the Cow Palace — are being transported by forklifts carrying 30 boxes on each pallet, according to reports. That works out to about 3,300 pallets total, or enough to cover most all of an NFL football field in file boxes to a depth of about four feet.

Things got even stickier when an initial investigation found thousands of documents missing. That prompted some curious efforts, including PG&E contacting thousands of former employees to ask if they’d ever “happened to take home” materials relevant to the investigation for work or study.

Unable to meet the March 15 deadline, the company petitioned the CPUC for an extension, saying that it would need at least through the end of 2011 to comply.

That prompted outcries from victims of the San Bruno disaster who were trying to get on with their lives, as well as utility watchdog groups, several of which suggested PG&E deserved to be fined up to $1 million per day it was out of compliance.

Not surprisingly, the CPUC’s proposed decision granting the extension the company requested, fining PG&E $3 million and threatening an additional $3 million fine if it failed to comply with the new deadline, left many incensed by what they considered a sweetheart deal.

The office of California’s attorney general voiced serious displeasure, both at the size of the fine and the lack of oversight the proposed ruling offered regulators. While the CPUC is scheduled to vote on the proposal by April 11, virtually any result is likely to face close scrutiny and possible challenges. Resolution of investigations and litigation related to the San Bruno explosion and fire are still years and at least tens of millions of dollars away.

EMC’s Captiva 6.5

An unusual situation? To be sure, but primarily in degree. Virtually every private and public sector organization maintains a delicate balance between managing physical and digital records, between supporting traditional and technological business processes.

Sometimes it takes an event like the San Bruno disaster to demonstrate just how far out of whack that relationship has become, but sometimes it takes much less.

So, can any IT solutions help companies effectively address and hopefully avoid such circumstances? Actually, yes, including EMC’s Captiva 6.5, the latest update of the company’s intelligent enterprise capture solution. Captiva 6.5 is designed to transform paper documents, faxes and other content into digital data used by business applications and IT processes. EMC benchmark testing found that a single Captiva 6.5 server can process more than 10 million images per day.

New customization, integration and management options notably simplify system deployment and reduce the time it takes for companies to begin using Captiva 6.5. In addition, the solution’s new Production Auto Learning feature enables systems to automatically classify and extract important business information from documents as they are being scanned, notably accelerating the capture processes for invoices, loan documents, policy claims, and many other document types.

EMC’s Captiva 6.5 is obviously aimed at global enterprises, but those are the organizations that have the greatest need for comprehensive intelligent capture solutions. They are also likely to be most aware of the potential costs they face — in legal fees and fines, wasted time and revenues and lost confidence and trust among customers and partners — should the unthinkable ever occur.

Taking such circumstances into consideration, and noting the painful ongoing lessons of PG&E’s San Bruno disaster, the potential return on wisely, proactively investing in a solution like EMC’s Captiva 6.5 is almost incalculable.

E-Commerce Times columnist Charles King is principal analyst for Pund-IT, an IT industry consultancy that emphasizes understanding technology and product evolution, and interpreting the effects these changes will have on business customers and the greater IT marketplace. Though Pund-IT provides consulting and other services to technology vendors, the opinions expressed in this commentary are King's alone.

Leave a Comment

Please sign in to post or reply to a comment. New users create a free account.

Related Stories

If your employer enacted a 4-day workweek with a fixed day off for everyone, which schedule would you prefer?
Loading ... Loading ...

E-Commerce Times Channels

New EU Law Will Force Google, Meta, Others To Expose Algorithms

The content of large tech companies, like Google and Meta, will be more tightly regulated under a new European Union law that received provisional approval Saturday.

Among the provisions of the Digital Services Act approved in an agreement between the European Council and European Parliament are transparency measures on the algorithms used by online platforms and services to recommend content and products to users.

“Platforms should be transparent about their content moderation decisions, prevent dangerous disinformation from going viral and avoid unsafe products being offered on market places,” Executive Vice-President for a Europe Fit for the Digital Age Margrethe Vestager said in a statement.

“With today’s agreement we ensure that platforms are held accountable for the risks their services can pose to society and citizens,” she added.

However, Daniel Castro, vice president of the Information Technology & Innovation Foundation, a research and public policy organization in Washington, D.C. noted that it remains to be seen exactly how the EU will implement some of the new law’s requirements.

“It’s possible that companies like Google and Meta are already meeting the DSA’s transparency requirements for ‘recommender systems’ under their existing disclosures to users,” he told TechNewsWorld.

“And these companies have also made progress in the past few years in terms of better explaining to users how they use their information and how their platforms work, such as ad transparency and ad library,” he added.

Focus on Big Tech

European Commission President Ursula von der Leyen explained in a statement that the DSA will upgrade the ground rules for all online services in the EU.

“It will ensure that the online environment remains a safe space, safeguarding freedom of expression and opportunities for digital businesses,” she said. “It gives practical effect to the principle that what is illegal offline, should be illegal online. The greater the size, the greater the responsibilities of online platforms.”

According to the European Council, the obligations introduced in the new law are proportionate to the nature of the services concerned and tailored to the number of users. Very large online platforms and very large online search engines — defined as services with more than 45 million active monthly users — will be subject to more stringent requirements.

To safeguard the development of start-ups and smaller enterprises in the internal market, the council continued, micro and small enterprises with under 45 million monthly active users in the EU will be exempted from certain new obligations.

“With the DSA, the time of big online platforms behaving like they are ‘too big to care’ is coming to an end,” Commissioner for the Internal Market Thierry Breton said in a statement.

Castro, though, maintained that the EU is making a mistake by focusing so much on the largest tech companies. “Smaller firms have a significant impact on consumers as well, and the largest companies are often the ones with the most resources and commitment to addressing harms,” he said.

Drag on Innovation?

Google did not immediately respond to a request for comment for this story, but in a blog written by Karan Bhatia, vice president for global public affairs and government relations, posted in October, the company warned, “While we support the ambition of the DSA to create clear rules for the next 20 years that support economic growth, we worry that the new rules may instead slow economic recovery.”

“They would prevent global technology companies like Google from building innovative digital tools like the ones that people have used through lockdown — and that will help European businesses rebuild their operations,” Bhatia wrote. “That would be a missed opportunity for Europe as it looks to the post-Covid future.”

In addition to algorithm transparency, other provisions empowering users and society include:

  • The possibility to challenge platforms’ content moderation decisions and seek redress, either via an out-of-court dispute mechanism or judicial redress; and
  • Access to vetted researchers to the key data of the largest platforms and provision of access to NGOs to public data to provide more insight into how online risks evolve.

Explosion of Public Scrutiny

“Even more impactful than making more information transparent about their algorithms is going to be the researcher data access provision,” observed Alex Engler, a fellow at the Brookings Institution, a nonprofit public policy organization in
Washington, D.C.

“There’s only so much you can learn by telling people something broad about a complicated issue as the interaction between an algorithm and millions of people who use it on a daily basis, but when you let professional researchers study all of that , they can come away with much more nuanced, specific understanding of what’s going on,” he told TechNewsWorld.

“Did a policy change lead to more disinformation? What are the mental health impacts of using social media?” he asked. “In those areas, we’ll see the most public scrutiny into large online platforms that the world has ever seen. Without any doubt, this will fundamentally change the level of public knowledge about these platforms.”

He explained that the DSA requires independent groups to validate what the companies are saying. “That gives them a lot less room to completely manipulate and hide the harms on their platforms,” he said.

He discounted concerns about the harm that opening algorithms to the public could have on the companies’ competitive edge.

“The competitive advantage of these companies comes more from their user base than the algorithms themselves,” he contended. “Facebook could tell me exactly how their algorithm works, and I wouldn’t be able to replicate the site because I don’t have billions of people coming to my website every day.”

Not Leaving EU

“The DSA is a significant piece of legislation, but it is unlikely to dramatically change the internet,” Castro noted.

“It imposes a number of new obligations on large online platforms, however, none of the rules are so onerous that large tech companies will leave the European market,” he added.

“A 450 million person market among the wealthiest countries in the world?” Engler asked. “I think the tech companies will comply with the new law.”

“These requirements are not so invasive that these companies won’t be able to make money anymore,” he said, “so I would be very surprised to see them leave.”

John P. Mello Jr. has been an ECT News Network reporter since 2003. His areas of focus include cybersecurity, IT issues, privacy, e-commerce, social media, artificial intelligence, big data and consumer electronics. He has written and edited for numerous publications, including the Boston Business Journal, the Boston Phoenix, Megapixel.Net and Government Security News. Email John.

Leave a Comment

Please sign in to post or reply to a comment. New users create a free account.

Related Stories
More by John P. Mello Jr.
More in Tech Law

Musk Masterminds Private Twitter Takeover for $44 Billion

Elon Musk, Twitter

One of the world’s wealthiest business moguls is putting his cash where his social media beliefs are. Twitter on Monday announced it entered into a definitive agreement for Elon Musk to purchase the social media platform for some $44 billion.

Once the transaction is completed Twitter will become a privately held company. Twitter made the announcement following Musk’s meeting with the 11-member board to discuss his offer to buy the social networking service.

The sale agreement gives the company’s stockholders $54.20 in cash for each share of Twitter common stock that they own upon closing of the proposed transaction. The purchase price represents a 38 percent premium to Twitter’s closing stock price on April 1, 2022, which was the last trading day before Musk disclosed his approximately nine percent stake in Twitter, according to Twitter’s announcement.

“The Twitter board conducted a thoughtful and comprehensive process to assess Elon’s proposal with a deliberate focus on value, certainty, and financing. The proposed transaction will deliver a substantial cash premium, and we believe it is the best path forward for Twitter’s stockholders,” said Bret Taylor, Twitter’s independent board chair.

Parag Agrawal, Twitter’s CEO, added that Twitter has a “purpose and relevance that impacts the entire world.”

Onward to Better Performance

Musk’s goal is to improve Twitter by enhancing the product with new features, he said in the announcement. He plans to make Twitter better by making its algorithms open source to increase trust, defeating the spambots, and authenticating all humans.

“Free speech is the bedrock of a functioning democracy, and Twitter is the digital town square where matters vital to the future of humanity are debated,” said Musk.

Twitter has tremendous potential. Musk said he looks forward to working with the company and the community of users to unlock that potential.

Accounting the Deal’s Purchase

The Twitter board unanimously approved the agreement for Musk to take over the controversy-laden media service. The transaction should close before this year ends.

For that to occur, Twitter’s stockholders must approve. Also needed is the consent of applicable regulatory approvals and the satisfaction of other customary closing conditions.

Musk has secured $25.5 billion of fully committed debt and margin loan financing. He is also providing an approximately $21 billion equity commitment. There are no financing conditions to the closing of the transaction.

Twitter officials will disclose the first quarter 2022 earnings results before the stock market opens on Thursday. In light of Monday’s pending transaction, Twitter will not hold a corresponding conference call.

Twittersphere All a Flutter

The Musk/Twitter developments dominated the news and social media posts all day Monday, offered Charles King, principal analyst at Pund-IT. Most of those discussions focused on Musk’s plans or intentions, including exactly what he means by “free speech.”

“From a technological perspective, I doubt any significant changes will occur in the short term unless adding an ‘edit’ button to tweet posts, streamlining the ‘blue check’ process, or adding a paid user tier qualifies as technology,” quipped King.

A larger tech-related issue is what, if any changes Musk is planning in regard to users’ data and how that might impact Twitter’s advertising strategy and ad sales, he observed.

“It is entirely likely that a more aggressive approach to leveraging that data could make Twitter more commercially successful and valuable. But such changes would also impact and likely deteriorate the experience Twitter users have come to enjoy and expect,” he told the E-Commerce Times.

Musk’s Motives?

Musk wants to go open source with algorithms that significantly define and automatically moderate the platform and reduce moderation. We are likely to see that the two parties begin to look at the code and aggressively complain about parts they think work against their political platforms, according to Rob Enderle, principal analyst of the Enderle Group.

Given that we think that anyone that disagrees with us is biased, even if the tool is not biased, these efforts will constantly create the impression they are, he added.

“I expect Twitter will bleed a lot of users and advertisers, and the resulting controversy will bleed over and damage Musk’s other companies over time, especially Tesla, which primarily sells to a liberal audience,” he told the E-Commerce Times.

“Added to this will be the founding belief that Musk is only doing this to stop moderation on his posts, which will undoubtedly be problematic for trusting Twitter to be unbiased in the future,” he said.

Lifting the Blinds at Twitter

Obviously, time will tell on how this is all executed, noted Casey Ellis, founder and CTO at Bugcrowd. Musk is a big fan of pulling back the curtains on how social media platforms and machine learning shape popular thought. He focuses on how and where these systems can be weaponized to cause harm, the implications of this on today’s society, and what improvements could be made to improve the integrity of information and news in the future.

“Bugcrowd’s work in election security has shown me time and time again the exploitability and outsized impact that platforms like Twitter are capable of, so an increase in transparency around the mechanics of how we all perceive what is true is fundamentally a good thing if Musk follows through on these aspects of what he has committed around the purchase,” he told he told the E-Commerce Times.

“His follow-through on this commitment with Tesla several years ago gives me some confidence that this will actually happen,” he said.

Jack M. Germain has been an ECT News Network reporter since 2003. His main areas of focus are enterprise IT, Linux and open-source technologies. He is an esteemed reviewer of Linux distros and other open-source software. In addition, Jack extensively covers business technology and privacy issues, as well as developments in e-commerce and consumer electronics. Email Jack.

Leave a Comment

Please sign in to post or reply to a comment. New users create a free account.

Related Stories
More by Jack M. Germain
More in Business