Alfred Music

Hey Travis, we talk yesterday about adding a sourcepath field in the Database on the Ftp, where you can point us the path where the files are stored, this is the file we are reading, please let us know if you can add it: ftp1.alfred.com/CatData/ALF_ALL_ALL_PipeDelimited.txt

alfred 24/4
Look asset if people are working on the same project
Once Alfred is up and running:
How team can suss out the assets and assign them to publications
requires manual workflow.
New version gives that workflow and makes is more easy.
Wants to work with eMedia mainly.
Servers might be out of date, PC server, wants to talk about it

Related products flavors... Digital PDFs for sale. New item numbers for sales reporting. Metadata. The artwork on a disk or mp3 are part of parent but tracked with other items. Internal and External Sellable vs Components
Workflow Need: Update all related items when one component has been updated
Import via REST API and Add pagination Add more search options? Maybe by file types
Do a New rsync and index fresh to Fix missing files issue on old server. Dont full text index

700G out of 18TB
Remove full text
Start ingesting

Hi Travis, we have made some good progress. The dropbox is downloading and it has about 500k files. It is very slow to download but is moving along. We also are testing the smart download and it is going well. We will want to get some feedback from you on that next time we meet.

Alfred is cancelled for this friday l.

Check on servers more.
Get drop box imported
Get Desktop Downloading working

hey! whats the status on the servers of Alfred? still running slow?

Alfred

1. Desktop tool for opening files
2. "Products" -> One to many "Production Assets" (individial files or folders)
2a. Add field to Product Assets table called Production Asset Type (old, primary etc)
3. Add Collaboration Folders
4. Script to append "Production Assets" from Dropbox by Item number in Product DB
5. Add a "Series" field to Products table (to filter by)


Print Prodution Archive Files
By product range
Assets/Product number

22GB

Search old volumes
Copy to old sub directory
Edit them and copy to new location

New SQL DB in the works on premise

Add filters such as Series
Publication Date ( to filter by years)

Harry potter

See web site

Add Collaboration

Add queue for downloads to show folders?

Download only and pick a folder

Script to append Production Folders from Dropbox by Item number

Production Folder maybe a file only... So rename to Production Assets
Add Asset Type such as "Old" or "Primary Folder"

Cristobal, could you check this?

Hey Dieneke,

Has tech@entermediadb.org accepted the invite from our dropbox archive? We aren't seeing it as accepted yet. Can you please let us know once you've accepted the invite so that we can adjust your access privileges? Thanks so much!

Travis C. Haase

Monday:
Exported: 3,942,295
Pending: 2,887,724
Complete: 1,052,958

So we did around 500K metadata-read & conversions during the weekend

Numbers before the weekend:
Exported: 3,942,295
Pending: 3,359,924
Complete: 581,321

If you want to do a debug sesh, I'm free for 3 hours.

Oh good to know, glad I got held off.

we just met Alfted this morning, we may hold on on the filters since we will get new fields from them

ok

credentials are admin/admin

Hi, I'd love to work on those filters for a bit if you'd like, I didn't have the log in and for some reason wasn't able to get ahold of you guys.

Related products
Book only
Book and CD
Same product but has a variant Component Look up table
Kit merging with CD

Add Drop box. Waiting

Filemaker DB has more DB


Called Products

Clean up dashes when searching 00-29 = 0029

Show only parents
Show lose asset

Add rights and royalties tab? With terriory? With contract? (custom app)

YES Add Components/Kits tab by Book# and CD Item Number, Audio. Player

Internal status field 3 char code
Share missing link

Add Primary PDF, Final Audio

Are realated Products actually products?

https://zoom.us/j/94713777672?pwd=Q1dhYmNCUld3Rkp2UkEwemY2WEJHdz09

Numbers today morning:
Imported: 3,942,295
137,000 Completed
3,800,000 Pending metadata

432,339 Conversions tasks, not sure if this is a good parameter to track

https://openinstitute.org/app/collective/projects/index.html?collectionid=AYtcWJUfjd8qDS3kA3Mq&nodeID=tasksAYtcWJUfjd8qDS3kA3Mq#asset-AYwYJEIzho8b7pMz17Wy

Report at the end of the day:

First Hot Folder (Production) imported 1,218,217
28,000 reported as completed
1,198,000 pending metadata

Conversion table is 75,000 records, It is doing that many conversions because looks like we do not have a lot of generated Media, I took a random Asset, listed the generated Media (untouched yet) and the Original folder, Original folder has 29 files, in 2018 we only generated files for 1 of them. (see attachment)

Add download Order queue to the UI. Have it download to the computer once it's ready. Need this for TAE as well.
Rebrand Media Boat -> eMediaLibrary App

Alfed 10/11
- How are you are looking up products now -> make that easier for clean up team
- Going through database they have now -> dropbox database, we want to connect to that. -> share directory with emedia. Read access is all we need.
- we will send an email adress to to connect tech@entermediadb.org, to set up dropbox
- a sample data instead of the dropbox folder could work too, screen full of results, a volume, 20 products -> travis send us
- We will do that in the nxt weeks and update the b-site and next time we meet we can show that to travis. Need to do a new reindex.
- Filter search is critical, add to collection is not used. Emedia is just primarly used as a search tool.

Goal = search files on dropbox database.
Desktop tool -> show in new future
Clean up -> after local server comes mid january

Problem now! Missing files on entermedia -> Files doesnt excist, we need to erase. Fresh scan of the files that we have.

https://zoom.us/j/94713777672?pwd=Q1dhYmNCUld3Rkp2UkEwemY2WEJHdz09
Were in there now

what I don't have a lot it is voice! hahaha

Cant open the meeting link, Chris is still in the meeting with Exensio I think

Yeaah!

Got it back

Cristobal lost power, christopher are you around?

https://zoom.us/j/94713777672?pwd=Q1dhYmNCUld3Rkp2UkEwemY2WEJHdz09

To all! Alfred weekly meeting in 20 minutes

Just whatsapp me and I'll hop on.

Hi, I can attend alfred meeting if you need an extra hand.

Alfred meeting at 10am ET

https://zoom.us/j/93114530188?pwd=bUFiVVRCSkRHeU9qUTlxaDVJUWVRdz09

Nice notes!

Assets - entermedia as an index, to find our assets,
Assets we difficult to parce, index and find the assets quickly.

Move of file folder, only work in entermedia, not possible anymore. just 1 person responsible for IT.

1 crash mac servers - 2 volumes crashes, no local backup.
No recovery for full server.

On premise server is coming!
Manually clean up files after that.
Once cleaned up: much better position to work better with entermedia, want to work and improve working with eMedia Workspace.
They have metadata for all products, database
Want to tie metadata + products in PIM system

Full time project, couple of months of clean up data.
What are going to do:
- Every folder with a status: in process, pending, completed to check of entire folder hierarchy.

Free standing file, itemnumber that doesnt belong to a folder, how to include them?

They are using a corporate account dropbox
- migrate everything from dropbox too entermedia eventually
Make project plan
dates
deliverables
- Upgrade existing site to this version
- see if that fix downloading / re import everything / we need a clean import.
- Open files and folders (warn for big folders 1gb) size limit. Progress indicator.
- Make folder hierarchy

GOAL: PRIO view all files on eMedia through download or open button.
Recommend:Add existing dropbox files and sync them up as well.

meeting soon for:
better workspace tool
Set up weekly meetings with another colleque
Waiting for IT group now, then we team up for the project. Get them workable in eMedia.

Friday calender - meeting next week.
Automating
Support your team
- meeting alfred standard weekly.

Two volumes crashed, no local backup.

Problem
Keep losing files,
Way too many files.
Limited budget

Solution
Using Entermedia,
Organizing files with metadata and AI
Building a friendly and powerful research tool
And Centralized

Vision
Migrating to a faster server and migrating archives to emedia with dropbox integration, then fine-tune search features.

Effectively organizing and storing to search hundreds of thousands of publications in total (300,000)

The goal is to be able to select all of the assets associated and then import them Into the project and say it’s signed off on, that we found all of the files for the next person to come in. But this never came to fruition due to employee turnover and layoffs. He’d like to continue with this vision

Important part: That we’re not missing out on individual files outside of folders and top levels.

Tasks:
Do more frequent scans.
Optimization and how eMedia can help:

* Check to make sure thumbnail conversion is complete. Making rescans more faster.
* Create an Automation tool for client.
* Working with 2023 version, adding features for uploading and downloading from your desktop version.
* Make it so you can see all folders related to each file, adding descriptions, metadata etc.
* Start building relationships using tags,
* We put together a B site to view. With organizational structures
* More user friendly approach to archives.
* The idea is so you can search from multiple places, by EG Product number, or other defining metadata.
* Attaching 1 to many production folders, attached to different places currently.

Where they’re at and how they’re going to move forward:
They’ve been working with Dropbox after the pandemic hit. IT department discovered that the VPN was terrible and grabbing files from server was absolutely terrible. No possibility of uploading updated assets to our server. New ITI doesn’t understand permissions.

1. Strategy is to have a list of specific products they want to clean up to start.
2. First steps is once IT has a file server up and running and migrated assets to server, That’ll be a good opportunity to start over and think about starting with a new index.
3. Focus on priorities first, then eventually clean up everything.
4. Start with a Reindex (R sync)
5. Migrate from Dropbox to Entermedia using integration
6. Include metadata:
7. Creation, modification, file name, size, location, type
8. Utilize Entermedia to clean up the assets. Will hire a team of 3 employees.
9. Have a thumbnail with supporting metadata for each asset

Challenges they’ve faced:

They’ve been backing up to amazon glacier, but had a huge problem with full server recovery after a major failure. They throttle download speeds so directories that were corrupted have been recovering since December of last year.

Highlighted pain points are mainly with file server upload/download speeds, VPN and need a better server.

Need better archives, to be in a better positioned to move towards intermedia for DAM and PIM solution


Quick updates on eMedia Side:
Working with a lot of clients in a similar situation,

Building a centralized standardized library is a common theme

We have a new version in development
And running it for a few large clients.

Travis’s insights:

1. Problem is assets are not linking on their current PIM system which is a problem.

When you search a product number:

1. Top level folder with product number then returns all individual files outside of the folders with that product number.

2. The search results only return top level directories, which helps.

3. But underneath he has all the files that live outside of the folders. The question is do those files want to be gathered? How do you narrow things down?
4. A tricky part of us, is that we don’t just do the books, we also have performance sets, which is their own animal (EG a concert band) Roughly 20-30 individual parts. But they have another record that represents that publication, but they also sell the score by itself. CB18361- But most assets they only sell one version. But it would be nice to relate and indicate other pub items that are related. EG single scores, or group scores to a concert band performance series.
5. There’s a lot of metadata that is useful to have eg: these files are from the 2023 concert band release, would be very useful. Question EG “Hey can you send me all of 2022s choral products?” Or “all of the harp music you have?” Or “all of the quartet music you have by Bach?” Or “all of the music you have by shoernburg?” Filter by tags. Add columns.
6.

Important part: Not missing out on individual files outside of folders and top levels. They want to utilize all the useful data they have.

https://zoom.us/j/97692876614?pwd=RWFISHFSTTNQbHRHZVE0RUYyZVZSQT09

I resend the google invites, but you need to accept it.

Alfrd prep meeting today at 10am CST / meeting is at 10.30 am CST
>https://zoom.us/j/97692876614?pwd=RWFISHFSTTNQbHRHZVE0RUYyZVZSQT09

I'm testing locally with a slice of 10 rows in my computer

File Download failed couple times yesterday, I ran it in an Screen so it finished that way

Hello, Im testing the Lookups, looks like they are not linking the other tables

How did you do?

import org.entermediadb.asset.importer.BaseImporter
import org.entermediadb.asset.util.Row
import org.openedit.Data

class CsvImporter extends BaseImporter
{
/**
* This is an example of making a field lower case
*/
protected void addProperties( Row inRow, Data inData)
{
super.addProperties( inRow, inData);
//createLookUp(inSearcher.getCatalogId(),inData,"Division","val_divisions");
}

}


CsvImporter csvimporter = new CsvImporter()
{

protected void createMetadata(Header inHeader)
{
List newnames = new ArrayList();
for (Iterator iterator = inHeader.getHeaderNames().iterator(); iterator.hasNext();)
{
String header = (String) iterator.next();
if(header.contains(".")){ //TODO: What about english and spanish language imports?
continue;
}
if( header == "height")
{
header = "pubheight";
}
newnames.add(header);
}
inHeader.getHeaderNames().clear();
inHeader.setHeaders(newnames);

super.createMetadata(inHeader);
}

};
csvimporter.setSeparator("|".charAt(0));
csvimporter.setModuleManager(moduleManager);
csvimporter.setContext(context);
csvimporter.addDbLookUp("genre");

csvimporter.setLog(log);
csvimporter.setMakeId(false);
csvimporter.importData();

Lookup:
Instrument
Format
Genre
Level

Description -> longcaption?
Pub_Date -> Entity_Date
Site Title -> Name
Series can be a Entity
width and height renamed
pubitem -> PrimaryImage


Create Product Folder entity that is related to PubProduct
Add a way to set uploadsourcepath with Category searcher/browser
Setup hotfolder on new B site with no metadata. To re-create assets and category tree. Use all old thumbnails

I set up alfredB site (m37 was strange turned into messed up DNS) , also installation was not ok at the first try, but now it is working. Im trying to connect to ftp to see the list but not sure if we require credentials

Any progress on this friday? If no we have to prepare this before the meeting on wednesday.

Hi Cristobal! That should be ftp1.alfred.com signing in with guest access. We have a few data files here, but the full inventory one is in this location:ftp1.alfred.com/CatData/DailyAll/ALF_ALL_ALL.txt

Meeting planned demo eMedia Workspace
Set up B-site for travis to show demo this week 23-27 october