Major industries, including finance, IT, industrial and government sectors, report over two critical security incidents with direct human involvement per day, new research from Kaspersky shows.
The Managed Detection and Response Analyst Report for 2023 details that more than one in five (22.9%) of high-severity incidents in 2023 were reported by the government sector, closely followed by the IT sector (15.4%).
The financial industry is less hard hit at just 14.9%, with industrial companies suffering just 11.8% of incidents.
Human coordinated attacks most effective
Almost one quarter of critical security incidents during 2023 were perpetrated with direct human involvement, with the most popular living-off-the-land attacks utilising powershell.exe, rendll32.exe and msiexec.exe.
In terms of MITRE ATT&CK techniques, phishing, account manipulation and exploitation of remote services were the most popular techniques used by attackers. The mean time to report for high severity incidents ranked at 36.37 minutes, with medium and low severity incidents taking 32.55 and 48.01 minutes respectively.
Speaking on the results of the report, Sergey Soldatov, Head of Security Operations Center at Kaspersky said, “In 2023, Kaspersky detected a smaller number of high-severity incidents, but observed a simultaneous increase in the number of medium and low severity ones. This redistribution of occurrences is associated with the detection of malware without visible traces of active human participation in attacks, which can be explained by the “commoditization of tools”.”
“However, it’s important to understand that the low number of high-severity incidents does not necessarily indicate low damage. Targeted attacks are now planned more carefully, and become more dangerous. Therefore, we recommend the use of effective automated cybersecurity solutions managed with the help of experienced SOC analysts,” Soldatov said.
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
As IoT technology progresses, the question of how to power these devices, particularly in locations where reliable electrical sources are scarce, presents a significant challenge.
Researchers at the University of Utah’s College of Engineering have pioneered a new form of battery that could help solve this dilemma. The solution, which is at the proof of concept stage, comes in the form of a pyroelectrochemical cell (PEC).
Developed by associate professors of mechanical engineering Roseanne Warren and Shad Roundy, the integrated device harvests ambient thermal energy and converts it into stored electrochemical energy. This effectively creates a supercapacitor or battery, which could be ideal for IoT and sensor applications.
Low levels of energy
The device works by charging with changes in its surrounding temperatures, whether located inside a vehicle, an aircraft, or even underneath soil in an agricultural environment.
“We’re talking very low levels of energy harvesting,” Warren said, “but the ability to have sensors that can be distributed and not need to be recharged in the field is the main advantage. We explored the basic physics of it and found that it could generate a charge with an increase in temperature or a decrease in temperature.”
Whilst solar cells can provide an alternative power source of IoT devices, the practicalities often present issues. “In a lot of environments, you run into two problems,” said Roundy. “One is that it gets dirty over time. Solar cells have to be kept clean. So in these types of applications, they get dirty and their power degrades. And then there are a lot of applications where you just don’t have sunlight available. For example, we work on soil sensors that we put just under the top surface of the soil. You’re not going to get any sunlight.”
With the use of a pyroelectric composite material made of porous polyvinylidene fluoride (PVDF) and barium titanate nanoparticles as the separator in an electrochemical cell, the device’s electrical properties change as it’s heated or cooled. This action modifies the polarization of the pyroelectric separator. This shifting of temperatures in turn creates an electric field within the cell, moving ions around and allowing the cell to store energy.
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
Despite only producing up to 100 microjoules per square centimeter from a single heating/cooling cycle, this could be enough for the needs of some IoT applications.
The study, funded by the National Science Foundation, is the cover feature in the March 21 edition of the journal Energy & Environmental Science, published by the Royal Society of Chemistry.
Samsung sends out updates every month as a matter of routine. These updates are security maintenance releases that fix bugs found in Android and Samsung’s own software.
The April 2024 security update has already been released for many devices, but Samsung also rolled out another update across the European region earlier this week.
This is a critical update that was released for a wide variety of devices, including but not limited to the Galaxy S24, S23, S22, Galaxy Z Fold 5, Z Flip 5, and the Galaxy A54.
It’s likely that this update would be rolling out to more devices soon. Under no circumstance should you skip this latest Samsung update when it arrives on your device.
Update is important for continued access to emergency numbers
Samsung is rolling out this important network update which makes some changes to cellular carrier support. Support for additional bands has been added in some markets to cater to network providers’ requirements.
Samsung has also dropped support for select bands, as some carriers may stop using those bands altogether. For example, many carriers are looking to drop 2G networks, meaning that your device won’t fall back to the legacy network.
The reason why this is a critical update is because you won’t be able to call emergency numbers in those markets if you don’t install the update. It will ensure your device has connectivity on all of the network bands that carriers in the region are using, so that in the event of an emergency, you’re able to call the required emergency services.
Installing this update will also remove TDD 4G network support for devices bought in Germany, meaning those devices won’t connect to these networks in Germany, Belgium, Denmark, France, the Netherlands, Luxembourg, Austria, Poland, Czech Republic, and Switzerland.
Given that the vast majority of carriers in Europe use FDD 4G networks, the removal of TDD 4G network support for users who bought their phones in Germany won’t have much impact when they connect to the 4G networks elsewhere.
As it stands, this appears to be a Europe-specific update so it may not be released in other markets. Those who do get the update in Europe should waste no time in hitting that install button right away.
Hundreds of thousands of WordPress websites are vulnerable to a critical severity flaw which allows threat actors to upload malware to the site through a bug in a plugin.
As reported by BleepingComputer, Japan’s CERT recently found a critical severity flaw (9.8) in the Forminator plugin, built by WPMU DEV. The flaw, now tracked as CVE-2024-28890, allows threat actors to obtain sensitive information by accessing files on the server.
The researchers also said the flaw could be used to change the contents of the site, mount denial-of-service (DoS) attacks, and more.
No evidence of abuse
Forminator is a plugin that allows WordPress operators to add custom contact, feedback, quizzes, surveys, polls, and payment forms. Everything is drag-and-drop and thus user-friendly, and plays well with many other plugins.
WPMU DEV has addressed the issue and released a patch. Users are advised to apply it and bring their Forminator plugin to version 1.29.3 as soon as possible. At press time, the WordPress.org website shows at least 500,000 active downloads, of which 56% run the latest version. That leaves at least 230,000 websites that are possibly still vulnerable.
So far, there is no evidence of CVE-2024-28890 being exploited in the wild, but given its destructive potential, and the simplicity to be abused, chances are abuse is just a matter of time.
While WordPress itself is generally considered a safe platform, its various plugins and add-ons present a unique opportunity for hackers looking for a way in. As a general rule of thumb, WordPress admins are advised to keep the platform, the plugins, themes, and add-ons updated at all times, and to deactivate all of the add-ons that they don’t actively use.
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
WordPress is the world’s number one website builder platform, with almost half of all websites on the internet being powered by the builder.
Microsoft and OpenAI are reportedly in the process of planning a groundbreaking data center project which would include an AI supercomputer named “Stargate”.
A report by Anissa Gardizy and Amir Efrati in The Information claims the goal of the project, which would be financed by Microsoft to the tune of over $100 billion, and which reportedly has a launch date set for 2028, is to reduce the two companies’ reliance on Nvidia, something that a lot of the tech giants involved in AI are increasingly looking to try to do.
Microsoft and OpenAI’s plan reportedly involves five phases, with Stargate being the fifth and most ambitious one.
The data center will be the supercomputer
The cost of the project is attributed to the age-old “sources familiar with the plans” (The Information says these are “a person who spoke to OpenAI CEO Sam Altman about it and a person who has viewed some of Microsoft’s initial cost estimates”), but neither Microsoft nor OpenAI have yet commented on the specifics of the project.
The new data center project is expected to push the boundaries of AI capability and could potentially exceed $115 billion in expenses. This is more than triple the amount Microsoft spent on capital expenditures for servers and equipment last year. Microsoft is currently working on a smaller, fourth-phase supercomputer for OpenAI that is expected to launch around 2026, The Information claims.
Shedding more light on the report, The Next Platform says, “The first thing to note about the rumored “Stargate” system that Microsoft is planning to build to support the computational needs of its large language model partner, OpenAI, is that the people doing the talking – reportedly OpenAI chief executive officer Sam Altman – are talking about a data center, not a supercomputer. And that is because the data center – and perhaps multiple data centers within a region with perhaps as many as 1 million XPU computational devices – will be the supercomputer.”
The Next Platform also says if Stargate does come to fruition it will be “based on future generations of Cobalt Arm server processors and Maia XPUs, with Ethernet scaling to hundreds of thousands to 1 million XPUs in a single machine,” and it definitely won’t be based on Nvidia GPUs and interconnects, which seems like a safe bet if the rumors are to be believed.
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
In 2023, the Securities and Exchange Commission (SEC) implemented new cybersecurity disclosure rules. These regulations mandate the disclosure of “material” threat and breach incidents within four days of occurrence, along with annual reporting on cybersecurity risk management, strategy, and governance.
The introduction of the new SEC cybersecurity requirements represents a critical milestone in the continuous fight against cyber threats. In 2023, chief information security officers (CISOs) revealed that three out of four companies in the United States were vulnerable to a material cyberattack. Consequently, cybercrime remains one of the foremost risks confronting US-based companies. Additionally, in the same year, nearly seven out of ten organizations in the United States experienced a ransomware attack within the preceding twelve months.
Cyberattacks pose significant risks to businesses, primarily in terms of financial damage. In 2024, cybercrime is projected to cost the United States alone more than $452 billion. Additionally, the loss of sensitive data is a consequential outcome of cyberattacks. In 2023, the United States ranked third globally in the percentage of companies reporting the loss of sensitive information.
Furthermore, data compromise incidents affected approximately 422 million individuals in the country in 2022, totaling 1,802 incidents. The US is recognized among the countries with high data breach density. Beyond financial and data loss implications, businesses are also wary of reputational damage, significant downtimes, and the potential loss of current customers, all of which can affect a company’s valuation and overall standing.
William Belov
Rise of awareness
Having in mind growing risks and new SEC rules, companies are strengthening their defenses, shows a recent report by Infatica, a provider in the proxy service market. According to the company’s data, the demand for proxy services searches has jumped by 106,5% over the last year. The reason behind this trend is proxies’ ability to imitate cybersecurity attacks. Therefore, using this technology companies can test their defenses.
The growing interest in proxy servers is not limited to seeking enhanced security measures alone. Searches for “free web proxy server” have risen by 5,042.9%, indicating a widespread pursuit for accessible solutions that offer anonymity. Meanwhile, the demand for “proxy server list” and “anonymous proxy server” has also seen significant upticks of 80.6% and 414.3%, respectively, highlighting the importance of reliable and discreet online operations.
While the SEC’s cybersecurity rules primarily target publicly listed companies, many of these firms depend on smaller third-party software and supply chain providers. A cyberattack at any juncture within this chain could result in significant consequences. This is why non-public entities are compelled to bolster their defenses too.
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
Major gap
As businesses ramp up their activities, significant gaps remain evident. A staggering 81% of security leaders acknowledge the impact of the new rules on their businesses. However, only 54% convey confidence in their organization’s ability to comply effectively. Surprisingly, merely 2% of security leaders have initiated the process of adhering to the new rules. Approximately 33% are still in the early stages, while a striking 68% feel overwhelmed by the new disclosure requirements.
Among the myriad challenges, determining the materiality of cybersecurity incidents stands out, with 49% of respondents highlighting its complexity. Additionally, 47% struggle with enhancing their disclosure processes, further complicating compliance efforts.
Here are several advices on how to prepare for complying with SEC cybersecurity rules:
1. Consolidate your cybersecurity risk data
With the new regulations mandating the disclosure of incidents upon discovery and comprehensive reports on cybersecurity strategy quarterly and annually, organizations must prioritize centralizing cybersecurity risk assessment and incident data. Consolidating this data into a single repository, rather than scattered across spreadsheet software or lost in email inboxes, increases the likelihood of meeting SEC deadlines and reduces the time spent gathering information from different departments and stakeholders for incident disclosure.
2. Acquire cyber risk quantification capabilities
Traditionally, organizations have used qualitative methods such as ordinal lists or red-yellow-and-green severity charts to assess the significance of cybersecurity incidents or other risk events. While the SEC recommends considering these assessments for incident materiality determination, quantifying cyber risk offers a more accurate insight into the financial impact of an incident. Understanding the quantified financial impact of cyber risks enables organizations to take necessary steps to mitigate costly risks or, ideally, prevent them altogether. This approach reduces the overall volume of disclosures required.
3. Optimize your incident management processes
It’s an opportune moment to conduct a comprehensive review of your organization’s incident management processes to ensure they are proficient in identifying, addressing, and reporting cybersecurity incidents. Streamlining and refining these processes facilitate the interception of cyber risks before they escalate into significant issues and enable swift reporting when necessary.
4. Enhance your cybersecurity and cyber risk governance
Ensuring compliance with the SEC’s new regulations involves adequately informing your board of directors about your organization’s cybersecurity risk management practices. Implementing robust reporting and communication processes is essential to regularly update leadership on cyber risk management efforts and any incidents experienced by the company. Furthermore, it’s crucial to articulate how these incidents may impact or are already affecting the organization’s strategy and finances.
5. Secure your third-party relationships
The updated regulations emphasize the importance of assessing cyber risk beyond the confines of your organization. Meeting the requirements for reporting on third-party cyber risk assessment and secure vendor selection underscores the necessity of establishing an effective third-party risk management program. Indeed, supply chain attacks aimed at smaller contractors and vendors frequently rank among the primary causes of cybersecurity incidents at larger organizations.
6. Improve a cyber risk culture within your teams
Digital transformation has significantly impacted nearly every organization, particularly in the years following the COVID-19 pandemic, which accelerated the shift of work and life online. Consequently, there has been a surge in employees connecting to organizational networks from various locations and devices, significantly expanding our cybersecurity attack surfaces. This shift underscores the critical importance of fostering a culture of cybersecurity risk awareness where cybersecurity is seen as everyone’s responsibility, not just the purview of the information security team. The more awareness of the threat posed by cyber risks that an organization can instill in its members, the stronger its overall cybersecurity posture will be, reducing the time needed to disclose incidents to the SEC.
While SEC regulations pose challenges, they also present opportunities. Following rules, can decrease the cybersecurity of the companies, enhance investor confidence, attract capital investment, and contribute to long-term business sustainability.
This article was produced as part of TechRadarPro’s Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro
This year, a critical agenda item for business leaders is the strategic consolidation of systems and technology stacks. The driving force behind this move? A quest to streamline operations, significantly cut costs, and mitigate risks. At the heart of this transformative strategy lies the challenge of addressing and minimizing technical debt associated with legacy systems.
Technical debt is a multifaceted issue, stemming from factors including the chronic underfunding of IT infrastructure, widening discrepancies between user needs and existing technological solutions, and a depletion of critical system knowledge as the architects behind these systems retire or move on. Anecdotes are rife of Cobol programmers being lured out of retirement with offers to more than double their earnings, highlighting the measures companies are taking to bridge these gaps.
These gaps not only make system maintenance management and upgrades a daunting and expensive task but also severely restrict an organization’s ability to adapt or extend its systems to tackle emerging challenges. This inertia acts as a significant barrier to innovation, curbing an organization’s agility and its capacity to evolve alongside its customers or to pivot into new markets successfully.
It’s not just software development that is experiencing these issues. It also spans the management of the system and environments in which they run, with a survey of 500 UK businesses revealing almost all have talent shortages in technology.
The scarcity of data experts is hampering efforts to wrangle maximum value out of data from inside and outside a business – a growing need among businesses as we move into the era of AI where data is a prime asset. Even for companies not on the brink of a major AI rollout, the necessity for fast access to reliable data for critical operations, decision-making, and crafting personalized customer experiences is paramount.
Chris Norton
Managing Director, InterSystems UK & Ireland.
A strategic imperative
Addressing technical debt transcends mere operational upkeep; it’s a strategic imperative that demands attention. As organizations evolve, decision-makers are tasked with a critical balancing act: determining whether to maintain or overhaul their aging tech infrastructure. This challenge is especially pronounced in sectors like banking and financial services, where each merger, acquisition, or launch compounds the complexity of existing systems.
Amidst this complexity, emerging technologies like data fabrics, cloud computing platforms with auto adaptive functionality that offer functional composability using no or low code, and Generative AI tools for coding, present opportunities to innovate. Yet, the rapid pace of technological evolution carries the risk of inadvertently accumulating new technical debt—a risk that, while difficult to quantify, looms large.
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
Creating a blueprint for modernization
The journey towards modernization necessitates a holistic approach that encompasses cultural and process changes, not just technological upgrades. Success in this endeavor, particularly in the face of a prevalent skills gap, requires a focus on initiatives that align with core business goals and promise a solid return on investment. Garnering widespread support hinges on clearly articulating the existing challenges that modernization aims to resolve, showcasing potential efficiency gains and bottom-line benefits.
Identifying the most pressing cases for modernization involves a thorough evaluation of existing systems, focusing on those that significantly hinder operational efficiency or degrade the customer experience. Key indicators such as system downtime, maintenance expenses, user feedback, and congruence with current business objectives are critical in this analysis.
Systems plagued by instability, demanding excessive manual oversight, or are incompatible with emerging technologies emerge as the top candidates for modernization. Targeting these areas for improvement not only boosts operational efficiency and customer satisfaction but also ensures that investments are directed towards initiatives with the greatest potential impact.
A focus on specific, high-impact areas will highlight early wins to a broad range of stakeholders, building momentum and support for ongoing modernization plans. For instance, upgrading an outdated customer relationship management (CRM) system might directly enhance customer experiences and sales, serving as a compelling proof point for the broader modernization agenda.
Allaying concerns over business continuity
The apprehension among senior leadership regarding the scale of modernization efforts and the potential risks to business continuity during the adoption of cutting-edge technologies is often over-played. Concerns that critical business data flows might be jeopardized are increasingly becoming outdated, thanks to advancements in data management technologies.
The evolution towards more sophisticated data management strategies, such as the implementation of smart data fabrics, is addressing these challenges head-on. This approach is particularly beneficial for organisations aiming for rapid access to high-quality, AI-ready data that is dependable, trusted, and accurate. This architectural approach underpins modernised systems without the need for replacement. The fabric brings the data together for analysis as required, without disruptive restructuring of the way information is held.
Using a smart data fabric, organizations can continue to extract value from their current systems, while concurrently advancing their modernization agenda. This dual-path strategy ensures that organizations can continue their operations without interruption, all the while laying the groundwork for a more agile, data-driven future.
Reducing cybersecurity vulnerabilities
As technology evolves, so do the tactics of cybercriminals, making it harder to protect legacy systems without another layer of data security that is easy to manage.
Reducing technical debt in this area will help organizations to significantly lower their risk profile, protect sensitive data, and streamline compliance with regulations. Automating and streamlining reporting through enhanced data management is a major gain in efficiency and compliance, boosting relationships with regulators as well as reducing cost. This is particularly vital in sectors such as finance, healthcare, and retail, where the integrity of data security and privacy is crucial.
Conclusion
In essence, tackling technical debt by consolidating technology stacks isn’t just about cost-saving; it’s a strategic imperative for businesses aiming to remain competitive and agile in an ever-changing landscape. Modernizing data management practices allows businesses to address the challenges associated with technical debt effectively, without the need for significant disruptions, paving the way for broader innovations across processes, products, and services.
This article was produced as part of TechRadarPro’s Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro
Chinese tech giant Tencent has agreed to make some of its most important apps available on Apple’s Vision Pro headset ahead of its launch in the country later this year, The Information reports.
Tencent is one of the biggest multimedia companies in the world and its apps and services are ubiquitous in China. It owns Weibo, WeChat, QQ, WeBank, Tencent Pictures, and games such as Honor of Kings and PUBG.
Apple TV+ and many other entertainment services such as Disney+ are not available in China. “Without Tencent’s contribution, Apple would have faced a greater challenge in marketing the Vision Pro in China,” the report explains, calling the company’s move “critical” to the device’s success in the country.
Apple is also apparently seeking Chinese partners to help it launch and run Apple TV+, Apple Arcade, and Apple Fitness+ in the country. The company has not launched any new services in China since iCloud in 2017 due to government restrictions.
iOS 18 will give iPhone users greater control over Home Screen app icon arrangement, according to sources familiar with the matter. While app icons will likely remain locked to an invisible grid system on the Home Screen, to ensure there is some uniformity, our sources say that users will be able to arrange icons more freely on iOS 18. For example, we expect that the update will introduce…
Apple today released macOS Sonoma 14.4.1, a minor update for the macOS Sonoma operating system that launched last September. macOS Sonoma 14.4.1 comes three weeks after macOS Sonoma 14.4. The macOS Sonoma 14.4.1 update can be downloaded for free on all eligible Macs using the Software Update section of System Settings. There’s also a macOS 13.6.6 release for those who…
The next-generation iPad Pro will feature a landscape-oriented front-facing camera for the first time, according to the Apple leaker known as “Instant Digital.” Instant Digital reiterated the design change earlier today on Weibo with a simple accompanying 2D image. The post reveals that the entire TrueDepth camera array will move to the right side of the device, while the microphone will…
Apple has previously announced three new iOS features that it said are coming to the iPhone later this year, as outlined below. The new features include the ability to install iPhone apps on the web in the EU, RCS support in the Messages app, and next-generation CarPlay. Web Distribution Apple recently announced that eligible developers will soon be able to distribute their iOS apps to …
Apple’s iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models concurrently, which is why we sometimes get rumored feature leaks so far ahead of launch. The iPhone 17 series is no different, and already we have some idea of what to expect from Apple’s 2025 smartphone lineup. If you plan to skip…
iOS 18 will feature a revamped Home Screen that is “more customizable,” according to Bloomberg’s Mark Gurman. He revealed this information in his Power On newsletter today, but he did not provide any specific details. Subscribe to the MacRumors YouTube channel for more videos. Apple will announce iOS 18 at its annual developers conference WWDC in June. Other features and changes rumored for…
We’re getting closer to the launch of new iPad Pro and iPad Air models, while rumors about iOS 18 are continuing to ramp up with this week’s surprise revelation that Apple has been talking to Google and others about potentially helping power the generative AI features expected to be a major part of this year’s update. Other news this week saw the release of iOS 17.4.1 and iPadOS 17.4.1…
The March 2024 edition of Microsoft’s Patch Tuesday is upon us, fixing dozens of vulnerabilities, including two critical severity issues which could result in remote code execution (RCE) and privilege escalation.
In its advisory, Microsoft announced addressing 61 CVEs, in addition to 17 Edge flaws fixed a few weeks prior. Of those 61 vulnerabilities, two are labeled critical, 58 important, and one low. The company said the flaws were not publicly known, or under active exploitation.
However, six were flagged as “exploitation more likely”, probably suggesting that they are relatively easy to discover and abuse, and that it was only a matter of time before a threat actor finds them.
Hyper-V flaws addressed
That being said, the two critical severity vulnerabilities are tracked as CVE-2024-21334 and CVE-2024-21400. The former has a severity score of 9.8, and is described as an Open Management Infrastructure (OMI) Remote Code Execution Vulnerability. The latter, on the other hand, has a severity score of 9.0, and is described as an Azure Kubernetes Service Confidential Container Elevation of Privilege Vulnerability.
Besides the two, other notable mentions include CVE-2024-21407, and CVE-2024-21408, two flaws affecting Hyper-V, and allowing threat actors not only to run RCE, but also denial-of-service (DoS) attacks.
This month’s Patch Tuesday also fixes a number of vulnerabilities discovered in products from other vendors, such as Adobe, AMD, Citrix, Chrome, NVIDIA, and many others. The full list of vulnerabilities serviced this month can be found on this link.
Every second Tuesday in a month, Microsoft releases cumulative updates, addressing as many vulnerabilities as it can (aside from critical updates which are released as soon as they’re available, and are usually known as out-of-bands patches). This is a longstanding practice in the IT industry that’s been picked up by many companies, including Adobe, and Oracle, and formalized in late 2003 by Microsoft.
Data lineage is a vital aspect of data management. It refers to the life-cycle of data, including its origins, movements, characteristics, and quality. This article delves into the concept of data lineage, exploring its definition, importance, role, and impact on various aspects of data management and business decision-making.
Data lineage can be described as the process of tracing and documenting the life-cycle of data, from its origin, through its transformation and usage, to its eventual storage. It provides a historical record of data, outlining its relationships and dependencies, thereby ensuring transparency and trust in the data. The importance of data lineage lies in its ability to provide visibility into the analytics pipeline. This allows organizations to understand how data is utilized and transformed across various business processes, enhancing the understanding of data flow.
Data Lineage Explained
Scott Buckles explains the importance of understanding and tracking the lineage of data, drawing parallels to the trust placed in the food supply chain. With automated data lineage tools, you can get real-time insight into data history, validate data accuracy, ensure regulatory compliance, and ultimately enhance trust and confidence in data. You wouldn’t eat food whose source you don’t trust. Why would your data be any different?
Other articles you may find of interest on the subject of AI fine tuning, training and data analysis :
Data Lineage and Its Importance
Definition and Importance:
Data lineage involves tracking data’s flow over time, including origin, changes, and destination.
Essential for validating data accuracy, consistency, and quality within organizations.
Relation to Other Concepts:
Data governance: Sets the structure for managing data, including ownership and policies.
Data provenance: Specifies the original source of data, often within the context of lineage.
Lineage is part of broader data management strategies, crucial for maintaining quality and standards.
Business Applications:
Key for decision-making and maintaining data integrity during migrations, updates, and error handling.
Provides an audit trail for granular troubleshooting and error resolution.
Documentation and Visibility:
Documents relationships between data across applications, detailing storage, responsibilities, and changes.
Tracks data generated by business users and systems, aiding in identifying changes and integration points.
Operational Mechanism:
Utilizes metadata to detail data attributes, helping users gauge data utility.
Integrates with data catalogs for better data discovery and mapping, aiding in ML algorithm development.
Use Cases:
Data modeling: Helps visualize data relationships, adapt to changes, and maintain accurate data models.
Data migration: Aids in planning and executing system migrations, and streamlines data system performance.
Compliance: Ensures adherence to data governance and privacy regulations like GDPR and CCPA.
Impact Analysis: Assesses the ramifications of changes within data ecosystems on reports and error exposure.
Data Lineage’s Role in Data Tracking and Ensuring Data Integrity
Data lineage is crucial in data tracking and maintaining data integrity. It allows organizations to trace data back to its source, ensuring the accuracy and reliability of the data. By offering a clear view of the data’s journey, data lineage helps identify any errors or inconsistencies in the data, ensuring data integrity. It also aids in data recovery, providing a roadmap to trace back to the original data in case of any data loss or corruption.
Data Lineage and Compliance with Regulations
In the current era of strict data regulations such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), data lineage has become essential for compliance. It helps organizations demonstrate the provenance and processing of data, a fundamental requirement for regulatory compliance. By providing a clear trail of data, data lineage enables organizations to prove that they are handling data responsibly and in accordance with regulatory requirements.
Data Lineage in Data Governance and Management
Data lineage is a key pillar of effective data governance and management. It provides a framework for understanding data flows, dependencies, and transformations, which is essential for managing data effectively. It aids in various aspects of data management, including data quality management, metadata management, and data privacy management, among others. By providing a clear view of the data’s journey, data lineage assists in making informed decisions about data usage, storage, and disposal.
The Impact of Data Lineage on Business Decisions
Data lineage significantly impacts business decision-making. By providing a clear and accurate view of data, it enables organizations to make informed decisions based on reliable data. It assists in identifying trends, patterns, and insights, which can drive strategic business decisions. Furthermore, it aids in risk management, as it provides a clear view of data flows and dependencies, enabling organizations to identify and mitigate potential risks.
Data lineage is a crucial aspect of data management that provides a comprehensive view of the life-cycle of data. It plays a key role in ensuring data integrity, compliance with regulations, effective data governance, and informed business decision-making. As data continues to grow in volume and complexity, the importance and impact of data lineage are set to increase further. This highlights the need for organizations to invest in robust data lineage tools and practices to ensure effective data management and informed decision-making.
Filed Under: Guides, Top News
Latest timeswonderful Deals
Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, timeswonderful may earn an affiliate commission. Learn about our Disclosure Policy.
