Strategies for Effective Data and Content Management

Discover essential strategies for effective data and content management, including indexing, storage solutions, toolsets, and cost optimization from an experienced media manager and Senior Solutions Architect. 

Introduction 

Data and content management is a critical concern for organizations of all sizes. Implementing effective strategies can significantly optimize storage capacities, reduce costs, and ensure seamless access to valuable media. Drawing from my experience as a media manager and a Senior Solutions Architect, this article will explore best practices for data and content management, offering insights and practical solutions to enhance your organization’s efficiency. 

Itemizing Your Indexes 

The first step in data or media management involves identifying the locations of your content and the appropriate tools for indexing and management. Utilizing an asset management system, which typically covers roughly 40% of your total data, whether structured or unstructured, is a common approach to managing the subset of media or content. To begin organizing your full data set, consider these questions: 

• What storage solutions are you using?
• What are the capacities and the organizational structure of these storages (e.g., volumes, shares, and directories)? How are they utilized?
• What are the costs associated with each storage per terabyte?
• What tools are currently in place for managing the data?
• How is content transferred and moved within your system?
• What retention policies are in place, and are they automated?
• What content is not managed by the Asset Management platform?

Answering these questions will set you on the right path toward effective management and cost optimization. Additionally, implementing measures like checksums during content indexing can help media managers quickly identify duplicate content in the storage, enhancing efficiency. 

Saving Your Toolsets 

Media management toolsets can vary significantly in their interfaces, ranging from Command Line Interfaces (CLI) to more visual interfaces like Finder or Asset Management UIs. Each interface offers a unique way to interact with and manage media effectively. 

Most Media Asset Management (MAM), Production Asset Management (PAM), and Digital Asset Management (DAM) systems feature Web UIs that support saved searches. These saved searches enable consistent content management across different teams and facilitate the sharing of management strategies. Implementing routine searches—whether daily, weekly, or monthly—is considered best practice in media management. For instance, during my time at a news broadcasting company in NYC, we used the term “Kill Kill Kill” to tag content for rapid removal. This industry-specific term signaled to everyone in production that the content was no longer in use. Although the word “Kill” might appear in a news headline or tagging field, it was distinctive in this triple format, making it a straightforward target for search-based content removal. This method efficiently reclaimed production and editorial storage space. 

Searches could also be organized by creation dates or hold dates to manage content systematically. Content older than three months was typically archived or deleted, and anything past its “hold” date by a week was also removed. 

For content like auto-saves and auto-renders in editorial projects, specific searches through a “finder”-like application were vital. Having a well-organized storage system meant we knew exactly where to look for and find this content. If content remained on physical storage but was no longer on the MAM, aka- “Orphaned”, it could be identified by its modified date. 

Using a CLI for content management is generally more complex and unforgiving, often reserved for content that was not deleted using other methods. This process should be handled solely by an administrator with the appropriate storage credentials. Preparing a list of CLI commands beforehand can significantly streamline the use of this interface. 

Maximizing Storage Efficiency and Minimizing Costs 

Just as nearly everyone has a junk drawer at home, organizations typically have their equivalent where users casually store content and documents, often forgetting about them. This leads to the gradual accumulation of small files that consume significant storage capacity. 

Assigning Storage Volumes 

To address this, organizations can benefit from assigning storage volumes or shares for specific uses rather than allowing open access, which helps prevent wasted space. For example, ensuring that only editorial content resides on the “Editing Share” simplifies the identification and management of caching and temporary files. 

Implementing Storage Tiering Policies 

Implementing a storage tiering policy for data at rest can also optimize production costs. By relocating less active projects to nearline storage, space is freed up for active projects. Many organizations differentiate between high-cost, high-performance Tier 1 storage and lower-cost Tier 3 storage, such as Production and Archive. Data that is not actively in use but should not yet be archived can remain costly if kept on Tier 1 storage due to its higher per-terabyte cost. For instance, if Tier 1 storage costs $30 per terabyte and Tier 2 costs $6 per terabyte, maintaining dormant data on Tier 1 can be unnecessarily expensive—$24 more per terabyte. This cost differential becomes especially significant in cloud storage, where monthly fees can quickly accumulate. Choosing a cloud provider with “free-gress” will also help control or enable costs to be predictable. 

Additionally, configuring alerts to notify when storage capacities are nearing their limits can help media managers prioritize their processes more effectively. These notifications also aid in reducing or eliminating overage fees charged by cloud providers when limits are exceeded. 

Refreshing the Evergreen 

“Evergreen content” refers to materials that are frequently used and never become obsolete, thus exempt from archiving. This includes assets like lower thirds, wipes, banners, intros, outros, and animations—items that are continually in demand. Such content benefits from being stored on nearline for swift access or on Tier 1 production storage, where it can be effectively managed with an optimized codec and bitrate to reduce its storage footprint while maintaining quality. The choice of codec is crucial here; graphic content that is originally rendered as lossless and uncompressed can be compressed before distribution to enhance efficiency and speed up access. 

Additionally, evergreen “beauty shots” such as videos of building exteriors or well-known landmarks should also be stored on nearline rather than archived. This placement allows for easy updating or replacement as soon as the content becomes dated, ensuring that it remains current and useful. Systems that allow for proxy editing should also use a strategy, where non-essential or evergreen content remains on the Tier 2 nearline. This ensures that content is housed at a cost effective and accessible space. 

Optimized Cloud Costs 

Cloud costs are a critical consideration in media management, especially with egress fees associated with restoring archived content, which can quickly accumulate if not carefully managed. Media managers can significantly reduce these costs with strategic planning. When content is anticipated to be frequently used by production teams, fully restoring a file is advisable. This will prevent multiple users from partially restoring similar content with mismatching timecodes. Additionally, carefully selecting a representative set of assets on a given topic and communicating this selection to production staff can streamline processes and reduce costs. 

For example, in the context of news, when a story about a well-known celebrity emerges, a media manager might choose to restore a complete set of widely recognized assets related to that celebrity. This approach prevents multiple users from restoring parts of the same content with different timecodes. Providing a well-chosen, easily accessible set of assets on a specific topic can prevent production teams from unnecessarily restoring a large volume of content that ultimately goes unused. 

Conclusion 

Each organization has unique production and data management needs. By strategically planning, defining, and organizing content lifecycles, they can streamline access to frequently used assets and minimize unnecessary expenses. Effective data and content management are essential for optimizing storage capacities, reducing costs, and ensuring unrestricted access to valuable media. Implementing diverse media management toolsets and defined retention policies facilitates organized archiving and retrieval, enhancing team collaboration and storage space optimization. By adopting these approaches and strategies, organizations can maintain a well-organized, cost-effective, and highly accessible data storage system that supports both current and future needs, ensuring seamless content management and operational efficiency.